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This  research  project  is  designed  co  examine  the  use  of  peer 
ratings  in  the  US  Array  generally  with  a  specific  possible  use  in 
assisting  in  the  selection  process  for  senior  service  college 
attendees.  Part  I  analyzes  the  feasibility  of  a  peer  rating 
system.  This  is  done  through  presenting  research  from  past  trials 
with  such  a  system  and  a  comparison  with  the  present  executive  type 
rating  system.  Part  II  tests  the  use  of  peer  ratings  in  selecting 
students  for  the  senior  service  colleges.  This  is  done  by  com¬ 
paring  infantry  officer  peer  ratings  with  the  actual  board  selection 
for  academic  year  1971-1972.  This  part  used  a  questionnaire  as  its 
prime  vehicle  for  research.  Conclusions  indicate  peer  ratings 
probably  cannot  be  used  widely  at  this  time.  Very  little  relation¬ 
ship  between  actual  peer  ratings  and  selection  board  results  for 
the  senior  service  college  was  found.  It  is  recommended  that 
further  work  be  done  along  this  line  with  the  goal  of  trying  to 
find  areas  where  valid  peer  ratings  might  be  used  if  it  can  be 
shown  that  they  are  superior  to  present  systems. 
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PREFACE 


The  Officer  Efficiency  Reporting  System  is  probably  the  most 
important,  tlough  possibly  the  most  controversial,  system  within 
the  personnel  field.  An  efficiency  reporting  system  which  has  the 
support  of  all  officers  is  vital  to  the  well  being  of  the  Army. 
Efforts  to  improve  the  system  are  continuous,  which  is  as  it 
should  be.  This  study,  which  examines  and  tests  the  Peer  Rating 
System,  might  possibly  be  of  some  use  to  individuals  working  in 
the  efficiency  reporting  area. 
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CHAPTER  I 


INTRODUCTION 

BACKGROUND 

The  military  services  are  searching  continuously  for  ways 
to  improve  the  officer  efficiency  reporting  system.  Although  the 
rating  forms  and  procedures  have  been  changed  six  rimes  since 
15«7,  there  remains  a  high  degree  of  overrating  which,  of  course, 
reduces  the  validity  and  use  of  the  reports.  Some  attention  has 
been  given  to  a  peer  rating  system  since  its  use  at  the  military 
academies  and  officer  candidate  schools  of  the  various  services 
has  proven  that  peer  ratings  have  a  high  validity  in  predicting 
the  future  value  of  officer  candidates  as  officers  in  the  military 
service.  In  addition,  some  success  has  been  indicated  in  studies 
of  middle  management  in  civilian  industry. 

PURPOSE 

The  purpose  of  Fart  I  of  this  paper  is  to  examine  peer 
ratings  and,  more  specifically,  to  address  the  following  questions: 
Could  peer  ratings  be  used  effectively  in  evaluating  all  officer 
personnel  in  the  Array?  Is  it  feasible-  to  add  a  peer  rating 
system  to  the  present  officer  efficiency  reporting  system?  Could 
peer  ratings  be  used  on  a  selective  basis  for  specific  type 
personnel  actions? 

Preceding  page  M 
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RESEARCH  PROCEDURES 


The  author  studied  several  books  and  articles  describing 
research  on  peer  ratings  and  their  use  in  performance  appraisal. 
Discussions  were  held  with  various  officers  in  the  grade  of 
Lieutenant  Colonel  and  Colonel  on  the  varied  aspects  of  such  a 
rating  system.  Since  very  little  research  has  been  completed  on 
this  subject  other  than  in  officer  candidate  schools  or  service 
academies,  much  of  this  paper  is  necessarily  based  on  opinions  of 
the  author  and  comments  from  a  number  of  fellow  officers.  However, 
it  we  are  really  interested  in  the  worth  of  a  peer  rating  system 
within  the  active  service  and  particularly  at  middle  and  higher 
management  levels,  a  beginning  must  be  made.  Questions  must  be 
surfaced  ana  the  subject  discussed  to  know  where  research  must  be 
accomplished.  Since  the  author  is  not  an  expert  or  better  still., 
possesses  only  limited  knowledge  in  tests  and  measurements, 
sociology,  and  psychology,  an  attempt  has  been  made  to  approach 
this  subject  from  the  practicability  and  reasonableness:  viewpoint. 

ORGANIZATION  OF  PART  I 

Part  I  provides  some  comments  on  the  present  rating  system, 
some  remarks  on  peer  ratings  in  general,  a  discussion  of  tfco 
feasibility  of  implementing  such  a  system  within  the  services, 
and  finally,  a  section  to  show  that  data  should  be  gathered  to 
determine  further  direction.  Special  emphasis  has  been  placed 
on  the  problems  which  can  be  expected  if  a  peer  rating  system 
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were  used  within  the  Army.  It  is  important  that  problems  be 
identified  early  so  that  they  may  receive  due  consideration  in 
any  future  research. 
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CHAPTER  II 


THE  t RESENT  RATING  SYSTEM 

IS  THERE  A  PERFECT  SYSTEM? 


Realize  the  limitations  of  the  knowledge  you  can 
obtain  under  the  best  of  circumstances  and  with 
the  nost  thorough  use  and  study  of  the  “data." 

When  you  deal  with  nations  and  systems  you  deal 
with  life,  the  life  remains  always  to  some  extent 
a  mystery,  and  refuses  to  become  “data."* 

Remember,  also,  that  man,  much  as  it  irks  him,  has 
not  the  gift  of  prophecy,  and  that  all  predictions 
are  therefore  essentially  unscientific  and  pre¬ 
carious.  ...  We  can  project,  but  we  cannot 
predict,  and  we  might  as  well  relax  about  it.2 

Remember,  finally,  that  political  science  is 
largely  an  art  ratt'er  than  a  science,  and  that 
therefore,  as  a  political  scientist,  you  are  some¬ 
what  in  the  position  of  a  painter  who  paints  a 
landscape,  or  a  novelist  who  depicts  a  family. 

No  artist  can  do  It  exactly  “right,"  and  the  same 
tree  painted  by  Cezeanne  or  Van  Gogh  or  Turner  will 
look  different.  Which  does  not  mean  that  any  of 
the  three  painted  it  wrong. 3 

Perhaps  it  is  unusual  to  begin  a  paper  on  efficiency  reports 
with  quotes  from  an  article  on  political  science.  But  if  one  will 
only  substitute  the  words  "officers"  for  "nations  and  systems"  and 
"efficiency  reporting"  for  "political  science,"  it  makes  the 
article  both  appropriate  and  somewhat  sobering.  If  one  will 
accept  the  theory  that  life  cannot  be  reduced  to  "data,"  that  one 
can  project  but  not  predict,  and  that  no  two  raters  can  ever  give 
the  same  exact  description  of  another  person,  th  tj  he  might  wonder 
if  maybe  the  efficiency  reporting  systems  over  the  years  within 
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the  military  services  have  not  been  pretty  good — even  with  all 
the  changes  that  have  taken  place.  Is  it  possible  that  the 
dissatisfaction  with  the  system  today  is  a  manifestation  of  the 
frustration  of  not  being  able  to  achieve  perfection  or  the 
impossible  in  ratings?  This  could  be  nearer  to  the  truth  than 
care  to  accept. 

DILEMMA  OF  THE  RATERS 

Problems  are  present  in  the  efficiency  reporting  system  of 
today,  but  none  is  as  important  as  the  dilemma  in  which  a  rater 
is  placed  each  time  he  completes  an  efficiency  report.  On  the  one 
hand  he  has  the  regulation  which  tells  him  there  should  be  an  even 
distribution  of  ratings  and  the  majority  of  officers  should  fall 
in  the  middle  ratings.  On  the  other  hand,  throughout  the  service 
an  average  rating  is  believed  to  be  the  "kiss  of  death"  as  far 
&s  future  schooling,  assignments,  and  promotions  are  concerned. 

This  may  or  may  net  be  true,  ’<ut  tfe  belief  that  it  is  true  is 
co mm,  among  most  officers .  And  irrespective  of  how  the  belief 
came  to  be,  it  is  nevertheless  a  fixed  and  firm  belief  which 
must  be  recognized.  If  the  rater  is  rating  an  officer  who  he 
Is  is  fully  capable  of  serving  at  the  next  hither  grade, 
what  should  he  do7  Should  he  follow  the  regulation  even  though 
he  Knows  inflation  in  ratings  is  spread  throughout  the  service?  Or 
should  he  rate  the  officer  in  line  with  what  he  believes  the 


trend  to  be?  This  is  a  problem  that  each  officer  wrestles  with 
each  time  he  completes  an  efficiency  report  and  this  Is  what  we 
should  be  worrying  about  more  than  the  fact  that  inflated  effi¬ 
ciency  reports  are  of  limited  value  to  the  Department  of  the 
Army  (DA).  The  refer  knows  the  future  of  the  ratee  is  in  his 
(the  racer)  hands,  not  in  the  hands  of  DA,  and  it  is  a  matter  ha 
cannot  and  will  not  take  lightly.  The  rater  is  faced  with  a  real 
dilemma.  The  problem  _s  net  that  the  decision  is  a  hard  one  to 
make;  the  problem  is  that  he  does  not  knew  in  his  own  mind  which 
is  the  right  thing  to  do.  Does  he  follow  the  regulation  to  the 
letter,  or  does  he  follow  what  he  believes  to  be  the  common 
practice?  It  is  obvious  from  the  inflated  reports  of  today  that 
the  titers  are  forced  to  rationalize  and  lean  toward  their 
responsibilities  to  the  ratee  and  not  the  system.  No  attempt 
should  be  made  to  change  this  attitude  among  raters.  The  system 
must  be  changed  s.mehow,  so  that  the  rater's  actions  in  fulfilling 
his  responsible ities  (as  he  sees  then)  toward  the  ratee  in  turn 
satisfy  the  system.  Until  this  happens,  there  is  no  solution 
to  the  rater's  dilemma. 


PEER  RATINGS 


As  work  continues  in  searching  for  a  uto^^an  rating  system, 
new  ideas  are  brought  forth.  One  of  these  is  peer  ratings.  It 
really  is  not  new  in  the  full  meaning  of  the  word  because  much 
work  has  bean  accomplished  in  this  area  already,  but  the 
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preponderance  of  reliability  and  validity  studies  have  been  with 
young  people  in  military  schools.  It  is  relatively  new  when 
considering  its  use  among  more  senior  officers. 

The  well  documented  success  that  has  been  achieved  through 
the  use  of  peer  ratings  in  several  instances  is  impressive.  When 
peer  ratings  are  added  to  a  rating  system,  the  ratee  is  seen  from 
a  broader  viewpoint — as  seen  by  his  equal  as  well  as  by  his 
superior.  Peers  seem  to  have  a  unique  ability  to  predict  the 
future  potential  of  an  individual  by  rating  him  on  certain  traits 
which  are  important  to  his  progression.  Peer  ratings  are  a  step 
toward  the  "whole  man"  concept.  The  theory  of  peer  ratings  is 
sound  and  the  use  of  such  a  system  by  the  Army  should  be  given 
full  consideration.  But  it  must  be  remembered  that  the  use  of 
peer  ratings  would  be  a  change,  therefore  it  must  be  handled  most 
carefully.  And  there  are  disadvantages  in  the  practical  applies-* 
tion  of  a  peer  rating  system  Which  must  be  considered.  In  short, 
the  Army  must  pursue  the  pros  and  especially  the  cons  of  a  peer 
rating  system  in  great  depth  before  any  decision  on  its  use  can 
be  made. 

The  peer  rating  system  is  one  more  step  in  search  of  a 
perfect  system— maybe  an  impossible  dream. 
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FOOTNOTES  FOR  CHAPTER  II 

1.  Konard  Kellcn,  "On  Problems  in  Perceiving  Other  Nations 
and  Systems,"  Security  Studies  Project  Number  15  (1968),  p.  24. 

2.  Ibid.,  p.  26. 

3.  Ibid. ,  p,  27. 
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CHAPTER  III 


PEER  RATINGS  IN  GENERAL 

BACKGROUND 

Many  articles  have  been  written  and  many  studies  documented 
concerning  the  advantages  and  disadvantages  of  peer  ratings.  The 
purpose  of  this  chapter  is  to  review  a  one  of  these  past  writings, 
with  particular  emphasis  on  what  are  believed  to  be  disadvantages, 
and  show  that  srore  research  must  be  accomplished  before  the  value 
of  a  oe-.r  rating  concept  in  a  "real  life"  situation  can 
be  determined. 

A  large  majority  of  the  actual  research  in  peer  ratings  has 
beer,  accomplished  in  the  military  services,  primarily  in  initial 
training  school  situations.  To  date,  the  emphasis  has  been 
placed  on  the  selection  process  ai.d  the  prediction  of  future 
potential  as  an  officer,  studies  of  graduates  of  the  Ucited  States 
Military  Academy,  United  States  Naval  Ac i demy,  Naval  OCS,  Air 
Force  XS,  and  Marine  Corps  OCS  have  all  proven  conclusively  that 
peer  ratings  are  the  most  valid  predictors  of  officer  performance 
than  any  ether  undergraduate  measures  (academic  and  instructor/ 
tactical  officer  grades).  The  validity  of  pear  ratings  is  well 
documented  in  Major  R  a»n  A.  Nadal's  Research  Report  No.  68-8 
which  reviews  a  substantial  number  of  past  studies  on  peer  ratings.* 
For  the  purpose  of  this  paper,  the  statement  that  peer  ratings  are 
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the  most  valid  racings  system  for  military  school  and  training 
situations  is  accepted  without  question. 

An  area  which  has  not  received  much  attention  in  past  studies 
concerning  the  military  is  the  use  of  peer  ratings  as  an  adjunct 
to  the  present  Officer  Efficiency  Reporting  System.  Would  peer 
ratings  more  accurately  predict  potential  than  ratings  by  superiors 
in  a  normal  duty  situation  the  same  as  they  predict  potential 
betc-c  than  the  tactical  officer  ratings  in  a  school  or  training 
situation?  One  of  the  conclusions  of  Major  Nad.nl ’ s  study,  based 
on  his  review  of  literature  by  recognized  authorities  on  evalua¬ 
tion  systems,  was  that  peer  ratings  are  the  most  valid  personnel 
racing  systems  now  available. 2  This  is  further  supported  by 
the  statement: 


In  both  military  and  civilian  environments,  in  terms 
of  the  findings  of  both  military  researchers  and 
oehavioral  scientists,  peer  ratings  predict  future 
success  better  than  tests,  better  than  subordinates, 
better  than  superiors,  and  in  some  cases,  better 
than  a  combination  of  all  three  put  together.-* 


The  above  quote  is  from  a  study  by  the  Office,  Deputy  Chief  of 
Staff  for  Personnel  (DCSPER),  Department  of  the  Axrey,  dated 


25  June  1969,  which  addressed  many  of  the  same  studies  as  Major 

Nadal,  The  DCSPER  study  atso  states: 

Two  main  arguments  against  peer  ratings  hove 
surfaced  in  study  group  discussions  and  interviews* 

.  .  .  and  two,  peer  ratings  are  applicable  only  in 
s  school  environment. 


The  school  environment  argument  assumes  that  peer 
ratings  are  to  be  used  as  a  weans  for  evaluating 
manner  of  performance  of  preseut  duties.  They 
aren’t.  Peer  ratings  normally  are  considered  es 
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predictors,  useful  in  estimating  potential. 

Therefore,  where  peer  ratings  are  made  is  of 
little  significance.  .  . 

Although  both  of  these  studies  are  very  convincing  in  their 
arguments  that  peer  ratings  will  predict  better  than  superior 
rating'*,  this  author,  based  on  a  .review  of  many  of  these  same 
studies,  questions  that  it  has  actually  been  proven  in  other  than 
a  school  or  training  situation. 


A  CASE  FOR  FURTHER  STUDY 


To  explain  this  doubt  referenced  above,  the  Air  Force  OCS 
study  by  Ernest  C.  Tupes,  1957,  should  be  examined  first. 

The  study  was  designed  to  investigate  the  relationship 
between  ratings  of  peers  in  an  officer  training  situation  on  a 
group  of  behavior  or  personality  traits  and  later  officer  per¬ 
formance  us  neasursd  by  officer  effic?.v--cy  aspects .  He  used  790 
graduates  ip.  this  test.  The  officer  efficiency  reports  covered 
a  period  varying  from  one  to  three  years.  His  conclusions  were 
that  there  was  little  doubt  that  personality  trait  ratings  (peer 
ratings)  as  obtained  are  related  to  later  successful 
officer  performance.^ 

Herein,  lies  the  basis  for  this  author's  questions.  What  was 
proven?  It  was  proved  that  peer  ratings  correlate  highly  with 
superior  ratings.  But  did  it  not  also  prove  that  superior  ratings 
correlate  highly  with  peer  ratings?  If  this  be  true,  then  once  a 
file  of  superior  ratings  is  accumulated,  could  you  not  predict 
future  potential  from  these  ratings  as  well  as  from  peer  ratings? 
Since  in  studies  reviewed  by  this  author  the  predictability  of 
peer  ratings  has  been  measured  against  performance  as  indicated 
by  superior  ratings,  or  in  civilian  research  against  the  promotions 


12 


received  as  a  result  of  management  "dev Lion  procedures,  it  has 
not  been  proven  that  peer  ratings  are  better  tools  for  predicting 
than  superior  ratings  in  a  normal  duty  situation.  It  would  appear 
that  today  we  could  say  with  complete  confidence  that  peer  ratings 
are  good  predictors  and  may  or  may  not  oe  better  than  supervisor 
ratings.  One  might  challenge  this  on  the  basis  that  peer  ratings 
have  been  proven  to  be  better  than  tactical  or  training  officer 
ratings.  It  is  submitted  that  an  individual  responsible  for 
making  evaluations  in  a  training  position  and  a  supervisor  on  the 
job  are  not  comparable  with  any  degree  of  accuracy. 

At  this  point  it  might  be  worthwhile  to  examine  the  argument 
that  "peer  ratings  are  only  applicable  in  a  school  environment" 
and  the  opposing  opinion  that  "where  peer  ratings  are  made  is  of 
little  significance"  referenced  above.  Again,  several  studies  are 
most  convincing  in  the  theory  that  peer  ratings  may  be  made  under 
any  circumstances,  but  others  lean  toward  the  argument  that  peer 
ratings  are  not  effective  in  some  situations.  One  such  study  was 
by  Albert  J.  Kubsuiy  as  outlined  below: 

The  author  was  interested  in  finding  a  more  realistic 
measure  of  student  quality  in  medical  schools.  Medical  research 
had  been  hampered  by  the  lack  of  such  measures,  particularly  in 
the  last  two  years  of  medical  school.  In  these  years  the  instruc¬ 
tion  is  less  academic  and  more  clinical.  The  instructors  see  the 
students  for  shorter  periods  of  time  and  have  less  opportunity  to 
draw  distinctions  between  students .  Tb«  students  have  more  time 
to  observe  each  other  candidly  and  comprehensively.  The  subjects 
were  87  medical  students  at  the  University  of  Pittsburgh.  Eleven 
factors  were  used  for  evaluation  such  as  "Medical  Facts,"  'Valm 
in  Emergency,"  and  "Community  Medical  Leader." 

Each  student  was  asked  to  name  three  classmates  who  would  be 
best  in  the  future  of  each  of  the  given  variables.  No  negative 
nominations  were  asked  for  and  rut  attempt  was  made  to  control  for 
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the  influence  of  slight  acquaintance.  Peer  nominations  were 
compared  with  the  evaluations  of  instructors  and  with  course 
grades.  Instructors  and  peers  agreed  fairly  well  on  some  factors, 
but  there  was  practically  no  relationship  in  other  factors.  The 
results  of  this  3tudy  suggested  that  evaluation  is  the  proper  role 
of  the  instructors  and  that  peer  nominations  should  not  have  a 
place  in  determining  class  standings  or  recoroeodations  for  posi¬ 
tions.  It  raised  the  question  abou..  the  place  for  peer  nominations 
in  a  medical  school,  and  indeed,  in  any  professional  school.  It 
was  Indicated  that  the  best  use  of  peer  information  might  be  for 
research  purposes,  especially  in  developing  more  comprehensive  or 
diagnostic  performance  criteria. 6 

This  study  is  cited  to  show  that  maybe  peer  ratings  are  not 
effective  in  all  situations.  Granted,  the  peer  ratings  in  this 
c%tse  were  measured  against  instructor  ratings  and  academic  grades 
instead  of  actual  performance  at  a  later  date,  but  the  fiucings 
cannot  be  overlooked  as  there  will  always  be  a  fine  line  between 
measuring  present  performance  and  potential  (future  performance). 
Also,  it  should  be  noted  that  in  the  last  two  years  cf  medical 
school  the  students  were  somewhere  between  the  school  or  closely 
supervised  training  environment  and  actual  job  performance.  In 
the  opinion  of  this  author,  it  gives  some  indications  that  as  the 
setting  moves  closer  to  job  performance  the  value  of  peer  ratings 
may  decrease. 

Perhaps  peer  ratings  would  be  a  valuable  asset  to  the 
present  rating  system,  but  before  it  will  be  accepted  by  the 
laity,  it  must  be  proven  beyond  doubt  that  peer  ratings  are  better 
predictors  than  ratings  by  superiors  Some  may  be  convinced  this 
is  fait  accompli,  however,  it  is  hoped  the  pieceeding  questions 
show  that  there  remains  some  basis  for  doubt. 
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Then  how  can  ii  be  proved  that  peer  ratings  ere  the  test 
predictors?  how  ear>  It  be  shown  chat  it  would  be  advantageous  to 
a  promotion  selection  board  to  have  peer  ratings  in  addition  to 
supei visor  ratings? 

One  method,  although  tlae  -oiisuaing,  would  he  to  nave  peer 
racings  and  supervisor  predictions  being  made  under  the  same 
circumstances.  Another  method  on  a  short  term  basis  would  be 
to  have  a  peer  prediction  for  the  results  of  some  type  selection 
board.  Either  of  these  type  method?  would  serve  as  a  beginning 
anr  would  provide  soma  insigh:  to  the  situation. 

In  the  first  method,  a  group  of  selected  officers  could  be 
rated  for  say  a  three  year  period  by  iheiv  peers  in  an  active  duty 
situation,  ’hiring  the  same  period  the  supervisor  could  subset 
prediction  ratings.  £w :>  though  the  present  efficiency  report 
form  has  an  area  for  measuring  potential,  it  is  believed  a  special 
fora  much  the  same  as  the  one  completed  by  the  peers  should  be 
provided  to  the  supervisor.  Then  at  a  prescribed  time  in  the 
future,  it  could  be  determined  who  had  made  the  best  predict  ion- - 
that  is  if  a  criterion  against  which  to  measure  could  be  estab¬ 
lished.  This  would  be  difficult  and  must  probably  would  have  to 
be  a  judgment  factor  based  ea  a  review  cf  records  by  a  selected 
group  of  officers.  Such  a  test  could  settle  the  issue  once  and 
for  all.  It  would  validate  or  invalidate  such  statements  as: 

Some  authorities  have  suggested  that  men  on  the 
same  level,  or  even  subordinates,  be  parties  to 
the  appraisal;  hut  in  view  of  the  facts  of  business 
life,  this  is  unrealistic.  Hen  of  the  same  level 
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are  apt  to  be  rivals  or  personal  friends,  and 
hence  biased  one  way  or  the  other.  In  any  case, 
if  George  appraises  Bill  and  Bill  appraises 
George,  logro'.ling  is  likely  to  insue,  and 
vitiate  the  whole  process. ? 

Such  beliefs  oust  be  refuted  «-”th  factual  evidence  before  a  peer 
rating  system,  will  receive  the  support  cf  officer  personnel,  with¬ 
out  which  it  is  doomed  to  failure.  Edward  A.  Rundquist  and  Reign 
H.  Bittner  state  that  "Merit  rating  systems  fail  because  raters 
are  not  sold  on  their  values, "8  They  further  stated  that  the  lack 
of  enthusiasm  bv  raters  can  be  caused  by  many  reasons  such  as: 

(1)  confusion  about  what  is  to  be  measured,  (2)  inadequate  train¬ 
ing  of  ratets,  (3)  th^  raters  having  no  part  in  the  development  of 
the  rating  procedure:- ,  (4;  management's  inadequate  understanding 
of  the  operating  problems  entailed,  (5)  confusion  on  the  purpose 
of  the  ratings,  and  (6)  laborious  procedures.9 

There  is  another  point  in  favor  of  a  test  such  as  described 

above.  The  peers  making  the  ratings  are  now  in  a  position  where 

ratings  have  a  greater  effect  on  such  things  as  promotion  and 

assignment— a  very  personal  thing.  To  quote  from  Reign  Bittner: 

As  soon  as  you  attach  direct  consequences  to  the 
merit  rating  system,  you  do  something  to  the 
ratings.  Raters  will  see  to  it  that  the  merit 
ratings  do  not  prevent  them  from  taking  action  they 
feel  desirable. ^ 

It  muse  be  learned  if  tarings  by  peers  do  change,  and  to  what 
degree.,  when  the  effects  of  their  ratings  definitely  have  a 
direct  bearing  on  their  own  future.  This  was  one  of  the  points 
addressed  bj  Hollander  in  his  study,  "The  Reliability  of  Peer 


Nominations  Under  Various  Condition?  of  Administration,"  as 
surasarized  below: 

The  study  was  conducted  with  some  700  officer  candidates 
at  the  US  Naval  School,  Officer  Candidate,  in  1955.  One  of  the 
variables  relating  to  the  reliability  of  peer-nominations  con¬ 
sidered  in  this  study  was,  "the  presence  of  any  differential 
effects  or  reliability  accruing  from  the  ust  of  forms  with  a 
'research'  set  as  against  those  with  a  'real'  set."  Half  the 
students  were  given  peer-nomination  forms  with  the  notation, 

"The  results  of  these  ratings  are  to  be  used  for  research  purposes 
only  and  will  not  affect  your  Navy  career,"  while  the  other  half 
received  forms  marked,  "The  results  of  these  ratings  may  be  used 
for  administrative  purposes."  His  conclusions  were  that  the 
differences  obtained  between  the  administrative  and  research  sets 
ware  minimal  as  regards  any  gain  in  reliability  use  of  one  as 
opposed  to  the  other.  He  further  concluded  that  results  obtained 
from  previous  peer-nomination  studies,  where  research  sets  were 
involved,  could  be  taken  to  have  "real  life"  implications.  How¬ 
ever,  he  did  introduce  a  caution  that  premature  conclusions  not 
be  drawn  regarding  the  differential  validity  of  forms  administered 
under  these  two  sets.** 

Although  the  results  of  this  study  indicate  that  peer  ratings  do 
not  change  in  a  "real  life"  situation,  it  still  is  speaking  of  the 
"real  life"  situation  as  it  exists  in  a  school  or  training  environ¬ 
ment.  Does  "real  life"  take  on  a  new  and  different  meaning  when 
-  individual  is  actually  on  the  joo?  The  primary  concern  of  a 
student  is  whether  or  not  he  will  graduate.  It  is  not  believed 
that  he  is  subjected  to  the  many  outside  pressures — social, 
financial,  moral,  and  professional — which  may  affect  his  work  as 
an  officer.  He  is  yet  to  learn  that  bad  decisions  can  affect 
such  more  than  his  grades.  It  is  doubtful  that  this  study  by 
Hollander  is  of  much  value,  other  than  as  an  indicator,  in 
determining  the  effectiveness  of  peer  ratings  for  officers  or, 
active  cuty. 
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The  second  test  could,  as  an  example,  have  peers  examine  the 
list  of  personnel  eligible  for  promotion  to  the  grade  of  Colonel. 
In  the  cases  where  the  peer  knew  the  candidate  well  enough,  he 
could  indicate  whether  or  not  the  candidate  should  be  promoted. 
The  results  could  then  be  compared  with  the  list  of  candidates 
actually  selected  for  promotion  by  the  selection  board.  This 
possibility  is  discussed  in  more  derail  in  Chapter  V. 
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CHAPTER  IV 


FEASIBILITY  OF  A  PEER  RATING  SYSTEM 

GENERAL 


The  previous  chapter  was  concerned  with  the  overall  concept 
or  value  of  peer  ratings  as  a  part  of  an  evaluation  system.  But 
regardless  of  how  good  peer  ratings  may  be,  there  still  remains  a 
question  as  to  how  feasible  or  practical  they  would  be  for  use 
within  the  military  service.  The  purpose  of  this  chapter  is  to 
present  a  detailed  discussion  of  the  areas  where  resistance  to 
peer  ratings  will  most  likely  be  encountered.  It  is  a  consolida¬ 
tion  of  as  many  problems  as  can  be  identified  from  a  review  of 
past  studies  and  from  discussions  with  fellow  officers. 

Problems  and  obstacles  could  be  expected,  the  same  as  with 
any  r.?w  system,  if  peer  ratings  were  added  to  the  present  officer 
evaluation  sysrem.  Certain  of  these  problems  are  real.  Others 
may  be  only  misconceptions  on  the  part  of  the  officers,  however, 
until  such  time  as  these  misconceptions  are  corrected  by  a  widely 
accepted  explanation,  they  will  be  an  obstacle  to  the  peer  rating 
system  just  the  same  as  if  they  were  real.  These  problems  must 
be  faced;  it  must  be  determined  which  problems  are  real  an*  which 
are  misconceptions;  and  it  must  be  determined  if  the  problems  can 
be  overcome.  Little  progress  will  be  made  until  this  is  accom¬ 
plished  .  jn fortunately,  some  of  the  problems  which  will  be 
discussed  are  already  present  in  our  rating  system  of  today. 
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However,  a  peer  rating  system  will  greatly  increase  the  number  of 
ratings  on  each  office*.  which,  in  turn,  will  magnify  any  existing 
errors.  Unless  they  '.re  corrected,  peer  ratings  would  only 
further  invalidate  the  ratings. 

WHO  IS  A  PEER? 

The  one  problem  which  will  be  contested  more  often  and  with 
the  most  vigor  is  that  of  determining  who  are  peers.  The  diction¬ 
ary  defines  a  peer  as  ''one  who  has  equal  standing  with  another." 
Unfortunately,  due  to  the  rank  structure  within  the  military 
services,  this  definition  hurts  much  more  than  it  helps.  Based  on 
the  rank  structure,  it  can  be  said  that  all  officers  in  the  grade 
of  major  (Air  Force,  Army;  Infantry,  Adjutant  General;  junior, 
senior)  have  equal  standing  and,  therefore,  are  peers.  But  can 
the  service  establish  a  peer  rating  system  based  on  this  definition 
without  further  defining  and  delineating  what  is  meant  by  a  peer? 
No!  However,  the  minute  efforts  are  made  to  precisely  define 
"peer,”  trouble  begins.  Everyone  will  have  his  own  idea — just  as 
everyone  presently  has  his  own  ideas  about  what  the  efficiency 
reporting  system  should  be--and  a  definitim  to  suit  all,  or  even 
a  majority,  of  the  officers  will  become  a  most  difficult  task. 

It  is  believed  all  officers  would  agree  that  a  more  definitive 
meaning  than  "equal  standing"  would  be  essential  before  attempting 
to  implement  a  peer  rating  system.  If  this  is  true,  then  why 
would  it  be  so  difficult  to  establish  an  acceptable  meaning — one 
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acceptable  to  all?  Perhaps  a  few  questions  at  this  point  can 
quickly  direct  attention  to  the  heart  of  this  problem. 

Take  the  case  of  two  majors,  one  with  one  month  in  grade  and 
seven  years  total  service  and  the  other  with  four  years  in  grade 
and  eleven  years  total  service.  Would  the  junior  major  be  quali¬ 
fied  to  submit  an  appraisal  on  the  senior  major?  The  senior 
major  would  in  ail  probability  answer  in  the  negative  and  he 
would,  without  doubt,  be  supported  by  a  large  majority,  if  not 
all,  of  the  officer  corps.  Would  they  be  correct?  It  might  be 
hard  to  say  that  added  experience  has  no  bearing  on  an  officer's 
ability  to  accurately  evaluate  traits  which  would  predict  future 
officer  performance.  But  if  the  junior  major  is  not  equally 
qualified  to  rate  the  senior  major,  then  when  do  majors  become 
peers?  When  they  are  within  one  year  of  equal  grade?  Two  years? 
It  is  a  hard  question  to  answer. 

The  next  point  which  repeatedly  comes  up  in  a  discussion  of 
this  subject  is  that  of  b::anch.  The  question  here  could  be, 

"Would  two  majors  of  equal  rank  and  service  and  serving  in  a 
division  headquarters,  but  one  an  infantry  major  serving  in  the 
G-3  Section  and  the  other  a  finance  officer  serving  in  the  Finance 
Section,  be  equally  qualified  to  rate  the  other?"  They  are  both 
serving  as  a  staff  officer.  Sven  if  it  could  be  shown  that  they 
were  equally  qualified,  it  is  doubtful  that  the  officer  corps 
would  ever  accept  such  a  condition. 
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Another  bruisirg  question  concerns  the  staff  officer  versus 
the  comnander.  Should  a  Lieutenant  Colonel  serving  on  the  divi¬ 
sion  staff  be  allowd  to  submit  an  appraisal  on  a  battalion  com¬ 
mander,  or  vice  veria?  There  are  strong  f^elinQs  in  this  area! 

It  is  evident  that  defining  a  peer  for  -my  peer  ratting 
system  will  require  considerable  thought  and  research.  Questions 
such  as  the  ones  surfaced  here  must  be  faced  squarely  and  studied 
in  detail.  Undoubtedly  this  problem  can  be  solved,  but  it  must 
be  done  with  great  care.  There  is  probably  no  other  point  which 
will  be  as  important  to  the  ratea  than  the  matter  of  i&o  his 
rater  will  be. 


PREJUDICES  AND  BIASES 

The  Problem 

The  subject  of  prejudice  and  bias  is  most  difficult  to  write 
about.  It  covers  a  broad  spectrum  and  any  writer  always  wonders 
if  he  is  not  allowing  his  own  prejudices  and  biases  to  influence 
his  writing.  However,  the  subject  cannot  be  avoided  in  any 
discussion  of  peer  ratings.  Whether  or  not  the  old  hang-up  that 
"peer  ratings  are  nothing  more  than  a  personality  contest"  is 
true  or  fictitious  is  of  little  significance  as  long  as  the 
officers  believe  it  to  be  true — and  the  number  who  object  to  a 
peer  rating  system  (supported  by  percentages  later  in  this 
chapter)  would  indicate  they  believe  it  to  be  at  least 
partially  true. 
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The  real  problem  is  that  of  the  studies  reviewed  by  this 

author,  none  has  proven  that  prejudices  and  biases  can  be  ignored 

in  a  peer  rating  system.  Those  individuals  who  are  in  favor  of 

peer  ratings  will  point  quickly  to  &  study  mads  using  187  Naval 

Aviation  Cadets  which  showed  that  friendship  had  no  bearing  when 

the  cadets  were  rating  other  cadets  on  leadership.*-  But  those 

individuals  opposed  to  peer  ratings  will  just  as  quickly  quote 

the  following  front  another  study: 

The  factors  observed  to  operate  most  often  to  give 
rise  to  prejudiced  judgement  were  these:  differences 
between  respondent  and  the  person  he  discussed  in 
nationality,  in  cirilian  or  military  status,  in 
military  rank,  in  social  status,  in  imaginativeness 
er  practicality,  in  types  of  assignment  (e.g., 
operational  job  or  desk  job),  in  attitudes  toward 
order,  regularity,  and  discipline,  in  tesrperament , 
and  in  drinking  habits. ^ 

And  so  the  arguments  continue. 


A  Democratic  Process 

What  are  some  of  the  prejudices  which  might  be  significant 
in  a  peer  rating  system.  One  which  probably  would  be  argued 
constantly  is  the  effect  of  source  or  type  of  commission — TJSHA, 
RA,  or  Reserve.  "Judges  rate  their  colleagues,  f?llow  students, 
or  fellow  teachers  higher  than  they  rate  others."-*  Sven  though 
the  problem  has  undoubtedly  diminished  during  the  past  seven! 
years,  can  it  be  stated  that  the  USK&  officer  nelly  3ees  a 
reserve  officer  as  a  colleague  in  the  true  sense  of  the  word? 

And  further  still,  does  the  Army  as  a  whole  look  at  them  equally': 
Do  they  receive  equal  consideration  in  assignments  or  does  it  run 
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in  crder  of  priority  from  the  USMA  to  RA  to  Reserve?  Does  one 
group  seem  tc  cluster  in  si&ilar  type  jobs?  Maybe  a  look  at  the 
present  setting,  the  U3  Army  T‘ar  College  (USAWC),  would  give 
some  insight  to  thes^  questions.  It  is  aa3uraed  that  an  assignment 
to  the  staff  and  faculty  of  the  USAWC  would  be  considered  as  a 
prestigious  type  assignment.  The  Officer  Biographical  Sketches, 
1971,4  shove  that  87  percent  of  the  ETC,  Army,  Combat  Arms, 
assigned  to  the  itaf f  sad  faculty  (for  this  survey  the  term  staff 
and  faculty  do;s  not  include  the  Operations  Group,  USAWC)  are 
USJiA  officers.  Even  Whan  considering  the  feet  that  the  USAWC 
student?  are  a  select  group,  the  percentages  run  much  lower. 

Only  31  percent  of  the  LTC ,  / rmy.  Combat  Arms,  assigned  as  students 
are  USMA  officers.  A  staff  ar.d  faculty-student  ratio  of  87  per- 
caot  to  31  percent  would  indicate  preference  is  given  to  USKA 
officers  in  assignment  to  the  USAWC .  Wily?  If.  there  any  special 
reason  or  did  it  just  happen?  Is  there  an  answer  vhich  would 
satisfy  the  officer  corps  as  a  whole?  Are  there  other  areas  of 
this  nature?  It  would  be  interesting  to  know  what  the  vatic  for 
high  level  positions  over  the  past  20  years  would  show.  The 
purpose  in  pursuing  this  line  is  not  to  question  whether  or  not 
this  is  right  or  wrong,  good  or  bad,  justified  or  unjustified. 

But  rather  to  emphasize  that  peer  ratings  are  a  democratic  process 
and  a  peer  rating  system  would  be  handicapped  until  such  time  as 
the  opportunities  ara  also  ox;  a  democratic  basis.  This  may  ba  a 
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minor  point,  but  it  could  be  one  of  those  small  barbs  which  help 
to  destroy  a  good  system. 


floral  and  Professional  Standards 


Moral  and  professional  factors  will  also  have  a  direct  bear¬ 
ing  on  the  validity  of  peer  ratings.  It  can  be  stated  that  it  has 
just  as  great  an  effect  on  superior  ratings,  but  this  is  doubtful. 
An  individual  studies  his  supervisor  and  is  able  to  adjust,  within 
the  limits  of  his  own  moral  and  professional  standards,  to  the 
stated  or  obvious  desires  of  the  "boss."  This  could  or  could  not 
be  a  desirable  characteristic,  but  it  is  necessary  for  an  efficient 
organization.  And  any  change  in  rating  systems  will  not  change 
this  attitude!  An  individual  will  not  act  in  violation  of  his 
supervisor's  desire  (again  within  the  limits  of  his  own  moral  and 
professional  standards)  even  though  he  might  believe  a  different 
action  would  improve  his  image  in  the  view  of  his  peers.  His 
"bread  and  butter"  is  too  closely  tied  to  his  supervisor.  In 
research  studies  carried  out  at  Michigan  Survey  Research  Center, 
one  curious  finding  appeared  consistently: 

It  was  discovered  that  effective  supervisors  tend  to 
have  bosses  who  follow  the  same  pattern  of  supervision, 
an*  that  ineffective  supervisors  tend  to  ha^e  bosses 
who  also  have  ineffective  patterns  of  supervision.  In 
other  words,  like  begets  like.  'He  tend  to  supervise 
as  we  are  supervised . '  Patterns  of  supervision  thus 
tend  to  flow  downward  in  an  organization.  The  reason, 
of  course,  is  that  rewards  and  punishments  flow  down¬ 
ward.  He  try  to  figure  out  how  our  supervisor  wants 
us  to  behave  and  then  modify  our  behavior  accordingly. 

He  tend  to  conform  to  our  supervisor's  expectation 
concerning  how  we  should  supervise.  .  .  . 
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We  tend  to  supervise  as  we  are  supervised,  not  as 
we  are  taught  to  supervise.  T"TT->  " 

if  this  is  true,  then  peers  do  not  always  see  a  peer  as  he  really 

may  be.  They  see  him  as  he  may  be  changed,  to  some  degree,  by 

his  supervisor.  Unless  they  (the  peers)  perceive  the  supervisor 

as  well  as,  or  in  the  same  manner  ns,  the  person  being  rated,  the 

validity  of  peer  ratings  will  be  lessened. 

Knowledge  of  Peers 

Along  this  same  line,  it  seems  to  be  a  common  belief  among 
advocates  of  peer  ratings  that  peers  are  usually  in  closer  contact 
with  what  the  ratee  does  hour-by-hour  and  day-by-day  and  that  a 
man  tries  to  present  only  his  best  side  to  his  supervisor,  but 
his  peers  see  him  as  he  is.  It  is  questionable  if  either  of  these 
beliefs  is  true  in  today's  environment — at  least  not  to  the  degree 
that  it  may  have  once  been.  It  is  not  uncosmon  that  an  individual 
spends  more  of  his  time  with  or  around  his  supervisor  than  his 
peers.  The  growing  size  of  the  Army  without  a  corresponding 
increase  in  on-post  housing  has  greatly  reduced  tne  amount  of 
time  officers  spend  together  after  duty  hours  (it  should  be  noted 
here  that  in  most  instances  to  date  where  peer  ratings  have  been 
used  effectively,  the  participants  all  lived  together).  As  stated 
in  the  Hilitary  Review: 

The  expanded  size  of  the  military  forces  also  has  had 
its  effect.  One  of  these  effects  is  the  limitations 
of  acquaintanceships  within  the  officer  corps.  A 
close  and  detailed  knowledge  of  one  another  is 
impossible,  and,  therefore,  reputations  are  vague 
and  limited.  .  .  .6 
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AI chough  this  author  was  describing  the  officer  corps  as  a  whole 
(and  even  felt  that  peer  ratings  might  help  to  solve  this  problem) 
it  is  believed  the  statement  will  apply  to  a  large  degree  to 
smaller  organizations,  e.g.,  post  or  division.  The  battalicn 
commander  simply  does  not  know  that  much  about  what  the  other 
battalion  commander  is  doing  in  many  cases.  Normally,  !  Stations 
axe  spread  out  by  organization  and  mission  much  more  than  in  the 
past.  This  author  discussed  this  situation  with  several  infantry 
officers  who  t\ad  served  as  battalion  commanders  in  a  division  in 
Vietnam.  They  were  asked  individually  if  they  felt  they  could  have 
made  peer  ratings  on  the  other  battalion  commanders  in  their 
brigade.  Without  exception,  the  answer  was  "No."  Each  one  stated 
that  he  just  didn't  see  the  other  commanders  that  much  or  know  them 
that  well  (unless,  of  course,  he  had  known  him  before).  They  all 
stated  they  knew  where  the  other  battalion  commanders  were  opera¬ 
ting  and  what  they  were  doing,  but  had  very  little  contact  with 
them.  Very  surprisingly,  two  of  these  officers  volunteered  the 
information  that  their  own  company  consanders  knew  even  less  about 
each  other  than  they  (the  battalion  commanders)  knew  about  other 
battalion  commanders.  They  stated  their  company  coonaaders  saw 
very  litcle  of  each  ether.  Even  in  Europe  the  battalions  are 
spread  out  so  that  many  battalion  commanders  do  not  see  much  of 
the  other  commanders.  The  days  of  the  old  regiment  are  gone! 

In  addition,  maybe  the  peer  doesn't  see  the  man  "as  he  is” 
better  than  the  supervisor.  There  seems  to  be  a  growing  trend 
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today  toward  an  individual  wanting  to  be  more  open  and  speak  his 
belief — a  trend  which  at  this  point  may  be  temporarily  upsetting 
the  Army  image  more  than  any  one  other  factor.  The  one  face  tc 
your  boss  and  another  face  to  your  peers  may  be  a  thing  of  the  past. 
Does  this  contradict  the  argument  used  earlier  in  the  chapter  that 
an  individual  will  not  act  in  violation  of  his  supervisor's 
desires  (within  limits)  because  his  "bread  and  butter"  is  too 
closely  tied  to  his  supervisor.  Maybe  to  some  degree,  but  not 
entirely.  An  individual  may  express  his  opinion  in  strong  terms 
to  his  supervisor,  but  in  the  end  he  normally  must  follow  his 
supervisor's  directions.  His  peers  may  or  may  not  know  of  the 
initial  position  he  has  taken  and,  therefore,  the  supervisor  may 
see  him  "as  he  is"  more  than  che  peers.  At  any  rate,  tines  are 
changing.  Studies  must  be  ^ade  to  determine  which  theory  carries 
the  most  weight  in  the  present  envj coiroent. 

Positive  and  Negative  Prejudices 
Studies  which  go  much  deeper  and  come  close.'  to  the  officer 
on  active  duty  must  be  accomplished.  Negative  prejudices  must  be 
examined  as  well  as  positive.  Hollander's  research  (referenced 
earlier)  to  determine  what  effect  friendship  played  in  peer  ratings 
would  be  considered  as  testing  a  positive  prejudice.  On  the  other 
band,  research  by  dejung  and  Kaplan  which  shewed  that  difference 
in  race  does  have  an  effect  when  individuals  are  rating  each 
other^  would  be  considered  as  testing  a  negative  prejudice.  More 
work  needs  to  be  done  to  determine  the  effect  for  persons  who  are 
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dislikec  for  some  reason — a  negative  prejudice.  Some  of  these 
problems  are  discussed  in  Assessment  of  Men.  This  book  is  an 
account  of  the  attempts  made  to  assess  the  merits  of  men  recruited 
for  the  Office  of  Strategic  Services  during  World  War  II.  It 
tells  how  individuals  were  selected  and  also  how  attempts  were 
made  to  measure  their  efficiency  as  a  worker.  Several  methods, 
one  of  which  was  peer  ratings,  were  used  to  measure  how  well  the 
man  performed  on  the  job.  This  situation  seems  to  be  more  closely 
related  to  the  active  Army  than  any  other  study  to  date.  Some 
of  the  findings  of  this  study  relating  to  social  factors  are 
as  follows: 


6)  ...  If  a  man  being  appraised  was  liked  or 

admired  by  a  respondent,  he  was  very  commonly  rated 
higher  on  all  traits  without  distinction.  On  the 
other  hand,  if  he  was  thought  to  be  low  in  one 
trait,  or  to  have  done  a  poor  job,  he  was  frequently 
rated  low  on  all  characteristics,  although  it  is 
unlikely  that  he  could  have  been  so  consistently 
undesirable.  Discriminating  distribution  of  trait 
ratings  on  a  single  individual  was  relatively  rare. 


8)  If  the  respondent  knew  the  individual  as  a 
co-worker,  he  may  easily  have  bod  a  different 
opinion  of  his  social  relations,  for  example,  than 
someone  who  knew  him  only  after  hours.  Re  may  have 
been  a  congenial  drinking  companion  and  a  crabby 
boss.  Each  view  might  have  been  correct  in  its  own 
limited  area,  but  it  was  hard  to  combine  them  into 
one  valid  over-all  rating. 

9)  ...  Similarly,  when  the  appraiser  and  one 
of  the  informants  _/ peers/  were  members  of  the  same 
clique  overseas,  but  another  informant  was  not  a 
member,  it  was  coonon  for  markedly  different 
opinions  to  be  expressed.  Of  course  both  such 
views  may  have  a  measure  cf  truth  in  them,  but  it 
is  not.  possible  to  represent  the  two  truths  in  a 
single  quantitative  rating.** 
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Prejudice  Toward  the  System 

One  danger  probably  greater  than  any  discussed  above  is  that 
of  the  officers  becoming  prejudiced  to  any  new  type  peer  rating 
system  before  it  has  been  given  a  fair  chance.  The  attitude  of 
the  officers  toward  peer  ratings  is  prejudiced  already  as  shown 
by  a  stuient  survey  at  the  Comand  and  General  Staff  College  (CGSC) 
in  1968.  This  survey  indicated  that  98  percent  of  the  students 
were  opposed  to  peer  ratings.^  The  reasons  given  by  the  students 
for  opposing  peer  ratings  were  many  and  varied,  but  were  closely 
related  to  prejudices  and  biases,  dissatisfaction  with  the 
selection  of  peers,  and  the  "personality  contest"  theory.  Here 
again,  as  with  Kubany's  study  described  in  Chapter  III,  the  CGSC 
study  was  concerned  with  using  peer  ratings  in  the  overall  evalua¬ 
tion  of  students  more  than  in  predicting  future  potential.  One  of 
the  problems  with  the  peer  rating  system  will  be  to  convince  the 
officer  corps  that  peer  ratings  are  measuring  traits  which  pre¬ 
dict  potential  and  not  an  evaluation  of  present  performance 
of  duty. 


HEASPREMvNTS 


The  quote,  "The  precision  of  an  instrument  cannot  properly 
be  measured  by  an  instrument  that  is  less  precise,"*®  leads  to  a 
very  intriguing  thought.  The  individuals  who  end  up  at  the  top  of 
a  peer  rating  system  have  been  placed  there  by  persons  evidently 
less  competent  than  they— a  most  unnatural  situation.  It  simply 
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seems  odd  to  think  that  persons  with  less  capability  could  predict 
an  individual's  future  potential  better  than  individuals  with  more 
experience  (the  supervisors).  It  is  not  intended  that  this  ques¬ 
tion  be  pursued,  but  it  does  open  the  door  to  other  problems— that 
of  determining  the  reliability  of  peer  ratings  based  on  the  ability 
and  conscientiousness  of  the  rater,  especially  in  the  case  of 
younger  and  less  experienced  officers.  It  could  be  argued  that 
the  same  holds  true  for  superiors  in  today's  rating  system,  but  it 
is  doubtful.  There  is  a  strong  feeling  of  responsibility  on  the 
part  of  supervisors  in  making  ratings  and  there  is,  without  doubt, 
much  time  and  effort  put  into  each  efficiency  report  rendered. 

The  system  of  an  indorser  and  reviewer  demands  this.  Can  this 
same  degree  of  a  feeling  cf  responsibility  ever  be  achieved  among 
all  peers?  This  weakness  of  the  peer  rating  is  noted  in  the 
Assessment  of  Men  as  follows: 

The  result*  of  these  techniques  /peer  ratings/  can 
be  immensely  valuable  or  completely  worthless 
depending  upon  the  degree  to  which  the  subjects 
can  be  induced  to  take  the  assignment  seriously 
and  to  execute  it  with  complete  honesty 
and  candor. H 

Striking  characters,  good  and  bad,  were  reported 
upon  more  frequently  than  less  colorful  figures, 
and  opinions  about  such  vivid  personalities  are 
especially  susceptible  to  the  errors  of  hearsay. 12 

Bittner's  article  touches  on  this  problem: 

.  .  .  merit  rating  is  in  competition  with  many 
other  things  for  the  rater's  time,  and  the 
accomplishment  r  '  these  other  things  has  a  more 
direct  bearing  in  the  rater's  mind  on  his  bread 
and  butter.  This  presents  a  dilenma,  because  an 


adequate  merit  rating  plan  requires  the  rater  to 
devote  considerable  time  to  it  if  the  results  are 
to  be  worthwhile.  .  .  .13 

This  lack  of  a  feeling  of  responsibility  may  be  a  danger  in  the 
event  of  no  indorser  and  reviewer  (who  obviously  would  have  no 
place  in  a  peer  rating  concept)  and  the  increased  number  of  reports 
which  would  be  required  of  all  officer  personnel.  Furthermore,  a 
peer  rating  system  would  be  a  system  with  almost  no  controls. 
Counseling  of  raters  in  a  peer  rating  system  would  necessarily  be 
in  general  terms  as  opposed  to  the  present  system  tftere  counseling 
by  the  indorser  and  reviewer  can  be  accomplished  on  specific  cases. 
Mandell  in  his  article,  ''Appraisals:  A  Valid  Management  Selection 
Tool,"  was  discussing  various  difficulties  to  be  overcome  to  make 
appraisals  more  meaningful.  One  finding  was  that  ratings  were 
more  accurate  when  obtained  orally  from  the  rater  by  a  trained 
staff  member. 34  Taough  this  would  be  near  impossible  in  the  Army, 
the  indorser  to  a  small  degree  and  the  reviewer  to  a  much  larger 
degree  presently  fill  this  role  of  a  trained  staff  member.  The 
fact  that  the  ■’■.‘'ports  of  *och  the  rater  and  indorser  are  reviewed 
by  a  more  senior  officer  adds  credence  to  the  present  system  and 
also  provides  the  ratec  with  some  feeling  of  protection. 

WEIGHT  VALUE 


To  establish  a  scoring  or  evaluation  system  for  the  users  of 
efficiency  reports  which  would  be  compatible  with  the  present 
system  will  be  hard.  If  s  peer  rating  system  were  implemented. 
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a  selection  boards  for  example,  will  now  have  many  more  reports 
for  review  and  it  would  seen  that  some  type  scoring  system  would 
be  necessary  (whether  present  selection  boards  do  or  do  not  use 
some  type  scoring  system  is  not  known  to  this  author).  But  before 
a  scoring  system  can  be  established,  the  relationship  between  peer 
ratings  and  superior  ratings  must  be  established— and  this  rela- 
tionship  should  be  common  knowledge  among  all  officers.  Will  the 
state  of  the  art  support  a  definitive  relationship?  Could  It  be 
determined,  for  example,  if  peer  ratings  should  have  a  weight 
value  of  40  percent  and  superior  ratings  a  weight  value  of  60 
percent?  This  will  certaiuiy  be  a  most  difficult  task,  expecially 
since  the  rating  of  an  individual  under  any  rating  sysfm  carries 
with  it  a  high  probable  error.  The  measurement  of  human  trait1? 
and  capacity  are  a  long  way  froat  the  exactness  which  we  are 
accustomed  to  expect  from  measurements . *5  xo  add  another  factor- 
peer  ratings  in  this  case— to  the  present  system  which  can  have  a 
bearing  on  promotions,  assignments,  etc.,  without  a  definite  and 
realistic  method  for  its  use  would  be  unsound. 


TRAINING 


Importance  of  Training 

Whac  is  so  important  about  training  an  individual  in 
techniques  of  efficiency  reporting?  The  rater  his  a  form  and  a 
regulation  explaining  how  Che  form  is  to  be  completed.  It  would 
seem  that  with  the  form,  the  regulation,  and  a  normal  amount  of 
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judg-^er.t,  any  officer  could  do  a  good  job  of  rating  another 
officer.  Evidently  this  is  not  true.  It  was  sobering  to  note 
just  how  many  times  the  word  ‘'training"  kept  cropping  up  as  a 
problem  in  the  various  articles  used  in  research  for  this  paper. 

The  training  problem  is  not  unique  to  a  peer  rating  system. 

It  is  without  doubt  a  big  weakness  in  the  present  rating  system 

and  is  probably  one  major  uaderlyirg  cause  for  the  number  of 

changes  that  have  been  squired  in  the  rating  system  over  the 

years— more  so  than  the  fact  that  the  reporting  form  being  used 

was  bad.  Bittner  addresses  this  specific  point: 

The  feeling  is  all  too  prevalent  that  the  way  to 
obtain  better  ratings  is  to  get  a  better  rating 
form.  It  is  not  surprising  that  this  feeling  is 
common,  for  if  a  tool  doesn't  work  it  is  natural  to 
look  for  a  deficiency  in  the  tool  rather  than  a 
fault  in  the  user  of  the  too?.  Nevertheless,  I 
believe  that  if  all  raters  were  properly  trained, 
almost  any  rating  fora  would  give  reasonably  good 
results,  providing  it  was  chosen  in  the  first 
place  to  conform  to  the  objective  of  the 
rating  program. 


It  is  interesting  to  note  that  there  is  considerable 
clamor  in  our  company  for  a  change  in  the  merit 
rating  form,  even  though  it  is  excreraely  doubtful 
that  our  form  has  ever  been  given  a  fair  chance  to 
prove  its  worth.  Mb  shall  be  obliged  to  change  the 
form,  too,  I  think,  because  sentiment  against  it 
has  crystalized  to  such  an  extent  that  systematic 
training  introduced  at  this  late  date  would  probably 
not  breakdown  the  resistance. 

.  .  .  Furthermore,  it  /training/  should  be  intr"»duced 
concurrently  with  the  Introduction  of  the  merit 
rating  systee  and  not  after  bad  training  habits 
have  been  formed— or,  what  is  worse,  after  resistance 
to  the  procedures  has  developed. 

As  stated  before,  training  is  not  a  problem  unique  to  a  peer 

rating  system  and  some  of  this  discussion  on  training  may  stray 


to  some  degree  from  the  subject  of  pee*:  ratings.  But  training  is 
believed  to  be  extremely  important  and  must  be  considered  in  any 
new  system.  Training  could  well  be  the  most  difficult  and  signi¬ 
ficant  factor  in  Implementing  such  a  revolutionary  system  as 
peer  ratings. 


Direction  of  Training 

What  should  be  the  thrust  of  a  training  program  for  efficiency 
reporting?  Perhaps  some  quotes  from  five  authors  will  highlight 
where  weaknesses  are  found.  From  Guilford:  "Individuals  differ 
in  the  capacity  to  judge  others."^  "Judges  do  much  better  if 
interested  in  the  ratings  they  make,"*®  and  "Different  judges  use 
different  criteria  in  judging  the  same  trait. "*9  From  Joseph 
Tiffin's  writings: 

Rating  people  is  a  skill  and  not  an  easy  skill. 

Good  production  men  and  even  good  supervisors 
are  not  necessarily  good  raters  unless  they  have 
been  taught  how  to  rate.  .  .  .20 

Walter  R.  Mahler  states: 

The  willingness  of  the  rater  to  formulate  accurate 
and  consistent  judgements  does  not  develop  auto¬ 
matically  nor  can  it  be  obtained  by  edict.  .  .  .21 

Gustav  C.  Hertz  writes: 

Plans  must  be  developed  for  securing  understanding 
and  acceptance  of  the  rating  program  by  both 
employee  and  supervisors . 22 

And  further: 

Merit  rating  is  a  comparatively  delicate  management 
device  and  one  which,  in  the  hands  of  untrained 
personnel,  can  cause  much  harm.  .  .  .  Ratings 
made  by  untrained  supervisors  are  frequently  the 
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soiree  of  appeals  and  other  types  of  efficiency 
rating  grievances  and  cause  the  system  itself 

to  fall  into  disrepute. 23 

And  finally  from  Roadman: 

.  .  .  some  attention  must  be  directed  to  the 
question  of  the  willingness  of  participants  to 
accurately  rate  each  other  if  they  know  full  use 
will  be  made  of  peer  ratings  for  administrative 
as  well  as  individual  development  purposes. 24 

It  is  obvious  from  these  quotes  that  the  three  key  words  in 
training  for  an  effective  rating  system  are  understanding, 
willingness,  and  acceptance — and  they  apply  equally  to  the  rater 
and  the  ratee.  It  is  debatable  how  well  these  three  factors  are 
integrated  into  the  present  rating  system,  but  however  hard  it  was 
to  get  to  the  present  situation — whatever  it  may  be— it  could  be  a 
more  complex  problem  to  mesh  these  factors  into  a  peer  rating 
system.  In  line  with  the  CGSC  survey  referenced  earlier,  another 
opinion  survey  of  some  1800  officers  showed  that  84  percent  of  the 
officers  favored  more  and  better  training  to  improve  efficiency 
reporting, 25  but  it  also  showed  that  78  percent  were  opposed  to 
being  rated  by  persons  of  equal  rank. 26 

The  number  of  methods  which  could  be  used  to  promote  under¬ 
standing,  willingness,  and  acceptance  is  a  study  within  itself, 
however,  seme  things  which  would  help  are  obvious  at  first  glance. 
One  of  these  is  "time.”  There  must  be  ample  lead  time  for  the 
personnel  in  the  field  «.o  digest  a  new  rating  system  before  they 
begin  submitting  ratings.  The  most  glaring  example  of  where  lead 
time  was  extremely  limited  was  when  the  present  form  67-6  for 
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efficiency  reporting  was  placed  in  effect.  The  new  DA  Regulation 
and  DA  Pamphlet  covering  this  reporting  form  were  received  in  the 
field  less  than  a  month  before  the  new  system  was  to  be  effective 
(based  on  experience  in  the  4th  Armd  Div,  USAEUR).  Administrative 
personnel  were  forced  to  prepare  lesson  plans  overnight,  present 
classes  when  they  themselves  were  not  sure  of  all  aspects  of  the 
new  system,  and  face  an  audience  which  was  openly  hostile  to  this 
sudden  change.  As  a  result,  the  efficiency  reporting  system  was 
chaotic  for  several  weeks  after  the  new  reporting  system  was 
implemented .  It  seems  rea. enable  to  assume  that  the  short  lead 
time  for  this  change  was  a  major  hinderance  to  its  acceptance  in 
the  field.  The  publication  and  distribution  of  all  material  con¬ 
cerning  a  major  change  such  as  this  should  be  a  minimum  of  three 
to  six  months  prior  to  implementation.  The  users  need  this  much 
time  to  establish  procedures  and  to  be  prepared  for  a  smooth 
transition.  Of  course,  publicity  explaining  the  reasons  for  the 
change,  etc.,  should  begin  long  before  this  three  to  six  months 
time  period. 

Another  place  where  training  must  be  emphasized  for  better 
understanding  is  in  the  meaning  and  value  to  terms.  One  interest¬ 
ing  experiment  in  this  area  is  summarized  as  follows: 

One  company  had  raters  put  20  of  their  workers  in  rank 
order  from  1  to  20.  Then  they  were  asked  to  down  the  list  and 
choose  the  person  they  considered  as  an  average  run-of-the-mill 
worker.  In  14  out  of  the  total  19  groups  rated,  the  raters  chose 
for  the  average  worker  the  persons  ranked  either  best,  second 
best,  or  third  best.  In  only  two  groups  did  a  rater  go  as  low  as 
seventh  best  for  the  average  worker.  Apparently,  "average"  to 
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these  raters  meant  outstandingly  good  and  defining  the  meaning  of 
"average"  did  not  clarify  the  point  for  them. 27 

There  are  presently  descriptive  words  on  the  Officer  Efficiency 

Report  form  which  are  more  confusing  than  "average."  Take  the 

word  "loyalty"  for  example.  A  rater  is  supposed  to  place  an 

individual  in  one  of  five  categories— top,  second,  middle,  fourth, 

or  bottom.  Many  will  argue  this  is  not  possible  as  an  individual 

is  either  loyal  or  disloyal— so  they  must  rate  him  top  or  bottom. 

Granted,  this  may  be  somewhat  extreme,  but  most  will  admit  it  is 

near  impossible  to  decide  when  a  person  should  be  rated  second, 

middle,  or  fourth  in  this  area.  Consequently,  most  raters  give  a 

top  in  this  block.  Is  this  wrong?  Row  can  you  define  "average 

loyalty?"  If  no  instances  are  known  where  the  individual  has  not 

been  loyal  (which  is  true  in  the  majority  of  cases) ,  then  how  can 

he  be  rated  below  top;  yet  the  efficiency  reporting  system  is 

supposed  to  be  such  that  the  "hump"  falls  in  the  middle.  Bittner 

touches  on  this  point: 

It  is  often  assumed  that  conmon  understanding  of  the 
meaning  of  a  trait  is  brought  about  by  defining  the 
trait  carefully.  An  interesting  finding  from  research 
on  ratings  in  our  company  bears  on  this.  He  have  found 
that  many  raters  cannot  even  distinguish  sheer  ability 
to  do  the  job  from  personality  despite  elaborate  defi¬ 
nitions  of  these  characteristics.  And  to  make  matters 
worse,  the  raters  were  rating  people  working  on  routine 
production  jobs  where  personality  was  not  a  component 
of  successful  performance. 28 

Another  example,  in  the  opinion  of  this  writer,  of  irtiere  two 
of  these  factors— understanding  and  acceptance- -are  made  difficult 
is  the  value  system  used  in  block  XII  of  the  Officer  Efficiency 
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Report.  Raters  are  supposed  to  rate  an  individual  between  0  and 
100  in  over-all  value  to  the  service.  Yet  grading  systems,  with 
rare  exceptions,  which  raters  have  been  subjected  to  all  their 
life  have  been  based  on  a  grading  system  of  70  to  100  as  passing 
and  below  70  as  failing.  Suddenly  in  the  Army  raters  are  supposed 
to  rate  an  average  officer  as  50.  We  are  all  greatly  hindered  by 
our  past  experiences  and  it  is  unnatural  to  rate  an  average 
officer  at  50.  If  it  is  necessary  that  50  be  average,  then  a 
training  program  which  is  far  superior  to  anything  heretofore 
used  is  necessary. 


ADMINISTRATION 


The  efficiency  reporting  system  is  an  administrative  burden 
at  all  levels.  Even  though  some  eight  percent2^  of  officer 
efficiency  reports  are  returned  by  DA  for  administrative  errors, 
efficiency  reports  are  probably  handled  throughout  the  Army  more 
carefully  and  watched  more  closely  than  any  other  administrative 
action  other  than  casualty  reporting — which  is  as  it  should  be. 
To  add  a  peer  rating  system  which  would  mean  many  more  reports 
could  create  a  real  administrative  problem,  even  though  the 
reporting  form  and  procedures  might  be  greatly  simplified. 
Control  will  always  be  a  problem.  The  value  of  a  peer  rating 
system  must  be  considered  in  light  of  this  increased  workload. 
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CHAPTER  V 


PEER  RATINGS  ON  A  SELECTIVE  BASIS 

A  PROPOSED  TEST 


In  Che  preceding  chapters  the  many  unanswered  questions— 
real  or  imagined  by  the  officer  corps— concerning  peer  ratings 
were  discussed  and  admittedly  it  painted  a  pretty  dark  picture 
for  peer  ratings.  But  it  by  no  means  shows,  nor  was  it  intended 
to  show,  that  peer  ratings  could  not  be  used  to  an  advantage 
sometime  in  the  future.  And  maybe  even  in  the  imnediate  future 
on  a  selective  basis  among  more  senior  officers.  Most  probably 
many  of  the  difficulties  discussed  would  be  eliminated  or  at 
least  reduced  if  peer  ratings  were  limited  to  more  senior 
officers.  With  maturity,  it  is  hoped  that  most  prejudices  would 
be  at  a  minimum  having  been  replaced  with  better  reasoning  and 
understanding.  Selective  use  would  not  flood  the  system  and  the 
reports  would  be  completed  more  accurately.  The  senior  officer 
would  be  better  motivated  to  do  a  conscientious  job.  At  any  rate, 
some  experimentation  must  begin  in  an  active  duty  environment 
simply  to  identify  specifically  the  problems  and  to  add  to  the 
knowledge  of  peer  ratings. 

International  Business  Machines  Corporation  (IBM)  conducted 
a  study  which  certainly  provided  encouragement  for  further 
research  in  peer  ratings.  It  is  suzsaarized  as  follows: 


The  purpose  of  the  6tudy  was  to  determine  how  well  peer 
could  predict  the  promotion  potential  of  middle  managers.  The 
setting  used  for  the  study  was  a  four-week  middle  management 
training  course.  The  students  were  broken  down  into  groups  of  16 
each,  and  worked  together  for  two  to  three  weeks  before  the  peer 
ratings  were  administered.  About  half  the  course  was  not  struc¬ 
tured  or  led  by  an  instructor,  which  allowed  for  much  student 
interaction.  They  all  lived  together  at  the  location  of 
the  school. 

Each  student  was  required  to  rate  all  students  within  his 
group  on  13  characteristics.  He  was  to  rate  them  on  a  scale  of 
1  to  5,  and  was  also  required  to  have  an  equal  number  of  ratees  in 
each  block— three  in  block  1,  three  in  block  2,  etc. 

Every  third  student  was  then  selected  for  further  observation. 
After  two  years  these  men  were  ranked  in  order  of  the  number  of 
promotions  received  since  completion  of  the  training  course.  This 
list  was  then  compared  with  the  mean  score  of  each  of  the  13 
characteristics  rated.  The  results  showed  peer  ratings  to  be  pre¬ 
dictive  in  10  of  the  13  characteristics.  Although  this  study 
cautioned  that  additional  research  is  necessary,  it  did  indicate 
that  peers  can  identify  those  middle  managers  who  will  move  into 
executive  positions. * 

Even  though  the  IBM  study  was  conducted  in  a  training  school 
environment,  rating  forms  were  not  signed,  and  the  participants 
were  not  in  a  "real  life"  situation— that  is,  the  ratings  would 
not  in  any  way  influence  their  own  or  their  fellow  students' 
promotion  potential— the  fact  still  remains  that  they  were  able 
to  make  accurate  predictions  in  this  situation.  It  seems  the 
next  step  would  be  to  change  the  setting  to  one  of  a  "real  life” 
situation  and  find  if  the  theory  remains  valid;  if  so,  then 
search  for  a  practical  application  of  the  theory. 

An  initial  test  to  provide  some  insight  to  the  subject  will 
be  Part  II  of  this  study.  The  test  will  be  to  match  a  set  of 
peer  predictions  against  the  actual  results  of  the  selection 
board  fcr  selection  of  students  to  attend  the  senior  service 
colleges  in  1971.  This  will  be  a  repeat  in  some  ways  of  the 
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IBM  test,  to  include  the  same  limitations,  moved  to  a  military 
environment.  However,  it  will  be  a  first  step  and  will  show  how 
much  difference  there  is  between  the  way  peers  see  an  officer  as 
compared  to  the  way  a  selection  board  sees  him  based  on  a  review 
of  his  records.  The  direction  for  further  research  will  depend  on 
the  answer  to  this  first  and  basic  question. 
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CHAPTER  VI 


CONCLUSIONS 

There  is  ample  evidence  to  show  that  peer  ratings  have  Leen 
most  effective  in  predicting  potential  in  training  school  situations. 
There  is  evidence  which  indicates  peer  ratings  Slight  be  effective  at 
middle  and  higher  management  levels  in  a  "real  life"  situation  in 
the  military  service.  *  But  at  the  same  time  there  are  many  unan¬ 
swered  questions.  Do  peer  ratings  predict  better  than  superior 
ratings  in  a  "real  life"  situation?  Would  it  be  too  difficult  to 
establish  a  suitable  definition  of  "peer"  in  an  Army  setting?  Can 
we  establish  conclusively  that  peer  ratings  would  be  of  any  signi- 
dicant  benefit  to  the  Army  if  added  to  the  present  rating  system? 
There  are  many  other  questions,  but  probably  the  biggest  question 
of  all  is,  "Is  it  practical?"  Can  it  ever  be  sold  to  the  officer 
corps?  The  theory  of  peer  ratings  may  prove  to  be  completely  sound 
in  all  situations,  but  this  will  be  of  little  use  if  we  are  unable 
to  establish  a  training  program  which  will  convince  the  officer 
corps  (which  already  believes  it  is  bad)  of  its  value. 

At  this  point  in  time,  peer  ratings  are  not  feasible  and  would 
serve  no  good  purpose  if  added  to  the  officer  efficiency  reporting 
system  for  all  officers,  or  even  on  a  selective  basis.  Rehashing 
over  studies  cf  the  past  will  not  solve  the  problem.  It  would 
appear  that  actual  testing  in  an  active  duty  setting  in  today's 
environment  is  the  only  way  to  obtain  the  answers  which  are  needed. 
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CHAPTER  I 


INTRODUCTION 

The  second  portion  of  this  study  project  presents  a  survey 
of  peer  ratings  and  subordinate  ratings  which  might  be  used  to 
assist  a  selection  board  in  selecting  officers  for  attendance  at 
the  senior  service  colleges.  It  is  intended  to  portray  a  practical 
example  of  the  peer  ratings  explained  in  part  one  of  this  study. 
Part  one  portrayed  some  examples  of  peer  ratings  conducted  in  a 
highly  controlled  popular  ion.  The  study  at  hand  is  a  real-life 
example  of  a  peer  rating  sysr.em.  An  opportunity  presented  itself 
after  the  project  was  commenced  to  obtain  subordinate  ratings  on 
the  same  group  of  officers,  hence  the  work  on  this  aspect  is 
also  included. 


PURPOSE 


For  many  years,  men  have  pondered  over  selection  lists, 
promotion  lists,  appointee  lists  and  any  other  lists  of  persons 
selected  by  ,  board  of  superior  officers,  and  asked  themselves 
why  certain  persons  were  or  were  not  selected,  promoted  or 
appointed.  These  persons  "knea"  of  several  glaring  errors  j.n 
omission  or  inclusion;  they  "knew"  these  errcrs  could  be  eliminated 
in  the  future  if  only  the  peers  and  subordinates  had  a  say  in  the 
selection  and  rejection  process.  This  basic  thought  describes 
generally  why  this  study  has  been  done.  The  hypothesis  of  this 
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study  is  that  pact  ratings  can  give  useful  information  to  the 
selection  board  from  a  heretofore  untapped  source.  As  Part  1 
indicated,  many  experiments  have  been  done  in  the  field  of  peer 
ratings,  as  well  as  practical  work  with  certain  selected  groups  of 
people.  This  portion  of  the  study  is  a  practical  example  of  a 
comparison  of  peer  ratings  (and  also  subordinate  ratings)  with 
executive  ratings.  The  percent  agreement  between  selection  and 
nonselection  by  peers  as  compared  with  the  selection  board  and  the 
same  comparison  between  the  subordinates  selections  and  the  selec¬ 
tion  board  selections  will  be  portrayed  in  Chapters  111  and  IV. 

SELECTION  OF  GROUP  FOR  STUDY 

Several  alternative  areas  presented  themselves  for  use  for 
this  practical  study.  These  alternatives  were  narrowed  to  three: 

1)  Current  selection  of  officers  to  the  grade  of  brigadier  general; 

2)  Current  selection  of  officers  to  the  grade  of  colonel;  and  3) 
Current  selection  of  officers  to  attend  a  senior  service  college 
for  the  academic  year  1971-1972.  The  first  two  alternatives  were 
discarded  primarily  due  to  the  sensitivity  of  early  information  as 
to  who  was  being  considered  for  these  promotions  and  the  fact  that 
these  promotions  are  so  singularly  important  in  the  officers' 
career  they  should  not  be  used  for  experimentation.  Another 
consideration  was  that  some  of  the  officers  who  would  be  involved 
as  raters  also  could  have  appeared  on  the  list  and  hence  be  ratees. 
The  third  alternative,  while  it  is  similarly  sensitive,  loses  a 
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good  deal  of  its  sensitivity  due  to  the  fact  that  no  one  performing 
the  peer  rating  could  possibly  appear  on  the  list  as  all  the  raters 
are  now  attending  the  United  States  Army  War  College  (USAWC). 
Sensitivity  was  further  reduced  by  virtue  of  the  fact  that  all 
eligible  infantry  officers  were  listed  on  the  questionnaire 
instead  of  on^.«  the  names  the  selection  board  uses.  In  this 
instance  the  selection  board  considered  only  259  previously  screened 
infantry  officers,  whereas  the  questionnaire  in  this  study  used  the 
entire  list  of  nearly  2000  eligible  infantry  officers. 

An  opportunity  presented  itself  early  in  the  study  tc-  gather 
subordinate  ratings  on  the  same  group  of  officers  for  which  the 
peer  ratings  were  done . *  For  the  remainder  of  this  study  project, 
only  peer  ratings  will  be  discussed  until  Chapter  IV  where  the 
analysis  of  the  results  of  subordinate  ratings  will  be  discussed. 
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1  Interview  with  Dandridge  M.  Malone,  LTC,  Staff  and 
Faculty  USAWC,  Carlisle  Barracks,  Pennsylvania,  October  1970. 
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CHAPTER  II 


METHODOLOGY 

This  study  project  was  conceived  in  September  of  1970.  At 
the  time  of  its  initiation  a  board  of  officers  was  in  session  at 
the  Department  of  the  Army  to  select  the  attendees  for  the  senior 
service  colleges  for  the  academic  year  1971-1972.  The  board  was 
scheduled  to  announce  its  results  sometime  during  the  early  part 
of  November  1970.  It  was  necessary  to  get  the  questionnaire  into 
the  hands  of  the  peer  group,  completed,  and  returned  prior  to  the 
list  of  selectees  being  announced  by  the  Department  of  the  Army. 

Due  to  this  timing,  the  entire  procedure  for  this  study  had  to  be 
expedited.  The  first  step  to  be  taken  was  to  designate  the  peers 
who  were  to  do  the  rating. 

In  Chapter  IV  of  Part  I  is  a  discussion  of  the  salient 
question  of  who  is  a  "peer."  For  the  purposes  of  this  study,  all 
US  Army  Infantry  Officers  presently  at  the  USAUC  are  considered 
peers  of  all  US  Army  Infantry  Officers  eligible  to  attend  one  of 
the  senior  service  colleges  during  academic  year  1971-1972.  Hence, 
included  in  this  definition  are  the  following  limitations:  1)  Both 
lieutenant  colonels  and  c-'lonels  rate  lieutenant  colonels  and 
colonels;  2)  Officers  with  from  fifteen  to  twenty-three  years  of 
conmissioned  service  are  rating  officers  with  from  fourteen  to 
twenty-two  years  of  commissioned  service  (years  of  conmissioned 
service  are  based  upon  the  criteria  for  attendance  at  a  sen .or 
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service  college);  and  3)  All  raters  and  ratees  are  infantry 
officers.  This  is  a  rather  broad  definition  for  a  peer  group. 
Under  the  conditions  existing,  it  appeared  to  be  the  roost 
workable  definition. 


SUBORDINATE  DETERMINATION 


For  the  purpose  of  this  study  a  subordinate  is  operationally 
defined  as  a  US  Army  Infantry  Officer  in  attendance  as  a  student 
at  the  Comnand  and  General  Staff  College  (C&GSC) ,  Fort  Leavenworth, 
Kansas.  These  officers  are  in  the  grade  of  major  or  lieutenant 
colonel.  It  is  possible  that  some  of  these  subordinates  may  in 
fact  be  slightly  senior  in  rank  to  some  of  the  officers  being 
considered  for  attendance  at  a  senior  service  college,  but  this 
would  be  an  inconsequential  number.  All  these  raters  from  the 
C&GSC  have  been  considered  to  be  subordinates. 

Something  should  be  said  at  this  time  about  the  selection  of 
the  infantry  officers  as  the  group  for  use  in  this  study.  Because 
the  Infantry  is  the  largest  branch  of  the  Army,  and  has  the 
greatest  representation  at  the  USAWC ,  and  at  the  C&GSC,  it  was 
felt  to  be  the  best  selection.  The  pros  and  cons  of  this  selec¬ 
tion  were  discussed  with  many  of  the  officers  of  many  branches 
before  the  fact,  and  most  of  them  agreed  it  would  give  the  best 
results  in  way  of  the  magnitude  of  the  response  to 
the  questionnaire. 
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The  next  largest  branch  represented  at  the  USAWC  is  the  field 
artillery  with  only  thirty-seven  officers  in  attendance.  It  was 
felt  this  would  be  too  small  a  body  from  which  to  get  sufficient 
results  for  the  purposes  of  this  study.  Ideally,  this  survey 
should  have  been  made  with  officers  from  all  the  branches  partici¬ 
pating.  Preparation  time  precluded  this;  there  was  time  for  only 
one  group  to  be  surveyed. 

LIMITATIONS  ON  NAMES 

Once  the  question  of  peers  was  settled,  the  next  preparatory 
step  was  to  secure  the  list  of  officers  who  were  eligible  to  be 
selected  to  attend  a  senior  service  college  in  academic  year  1971- 
1972.  It  is  beyond  the  scope  of  this  study  to  explain  the  prep¬ 
aration  of  the  rosters  of  officers  eligible  to  attend  a  senior 
service  college;  however,  an  excellent  explanation  was  given  by 
LTC  Francis  W.  Craig  of  the  Office  of  the  Deputy  Chief  of  Staff 
for  Personnel,  US  Army  during  a  personal  interview  in  November  1969. 
Suffice  it  to  say,  this  study  used  the  nearly  2000  infantry 
officers'  names  from  the  initial,  nonedited  list  of  over  7000 
names  of  officers  of  all  branches  of  the  Army  supplied  by  the 
Office  of  Personnel  Operations  (0P0)  at  the  Department  of  the  Army.*- 
An  attempt  was  made  to  secure  the  list  of  names  the  selection 
board  actually  used  which  contained  only  259  infantry  officers. 

Due  to  the  sensitivity  of  this  list,  it  could  not  be  released 
before  the  results  of  the  board  were  made  known.  The  selection 
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board  is  not  provided  the  entire  population  to  consider.  They  only 
consider  those  names  supplied  to  them  by  the  0P0.  Here  we  see  an 
informal  type  of  peer  rating  system  at  work,  (the  action  officers, 
(lieutenant  colonels  and  colonels)  in  the  branches  and  in  the  0P0 
selecting,  from  records,  those  best  qualified  for  the  selection 
board  to  consider!)  To  the  uninitiated  this  list  of  2000  names 
may  seem  long  and  unwieldy,  however,  neatly  printed  an  alphabetized 
it  proved  not  overly  difficult  to  read  and  to  complete.  As  an 
additional  consideration  in  its  ease  of  use  for  this  particular 
group  of  raters  is  the  fact  that  all  officers  are  accustomed  to 
perusing  long  lists  of  names  on  promotion,  selection,  transfer,  and 
qualification  and  other  lists  throughout  their  careers.  There  are 
a  few  areas  for  error  in  this  list  of  nearly  2CC0  names  which  will 
be  discussed  at  the  end  of  Chapter  III;  however,  the  errors  do  not 
seem  to  be  of  any  major  proportions  and  can  generally  be  discounted. 
They  are  pointed  out  for  completeness  of  this  study  and  as  a  cau¬ 
tion  for  anyone  undertaking  a  similar  study  in  the  future. 

PREPARATION  OF  QUESTIONNAIRE  AND  SCALES 

With  a  workable  definition  of  peers,  and  a  proper  listing  of 
officers  to  be  rated,  the  heart  of  the  study  was  in  preparing  a 
valid,  manageable  questionnaire  and  set  of  instructions  for  the 
raters. ^  This  is  deceptively  difficult.  A  copy  of  the  actual 
questionnaire,  with  only  one  page  of  names  (as  an  example  cf  the 
30  such  pages  of  names)  is  at  Appendix  A.  The  purpose  of  the 
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study  was  stated  as  simply  as  possible  without  trying  to  motivate 
the  rater  in  any  direction  except  to  complete  it  using  his  best 
judgment  and  mature  reflection.  It  was  purposely  designed  to  be 
as  simple  as  possible  and  to  be  completed  in  a  brief  period  of 
time.  A  great  deal  of  thought  and  trial  and  error  were  involved 
in  developing  the  association  scale  and  the  suitability  scale. 
Suitable  scales  for  these  purposes  could  not  be  found  in  the 
literature  which  was  available.  These  scales  have  no  known  pre¬ 
cedent  and  are  not  based  on  any  known  previous  study.  Perhaps 
future  work  can  refine  these  to  be  even  more  definitive  and  mean¬ 
ingful.  It  is  recognized  these  scales  may  have  different  meanings 
for  different  raters.  The  raters  voiced  no  difficulties  in  using 
the  scales  when  queried  about  them.  A  more  detailed  discussion  of 
the  use  of  these  scales  is  outlined  later  in  this  chapter. 

After  the  questionnaire  was  completed  it  was  pre-tested  on 
ten  officers  to  determine  its  adequacy.  Several  minor  changes  in 
wording  and  format  were  suggested  and  some  of  the  suggestions  were 
incorporated  in  the  ;inal  format.  Basically,  however,  the 
questionnaire  remained  as  initially  drafted. 

ADMINISTRATION  OP  QUESTIONNAIRE 

The  administration  of  the  questionnaire  was  done  in  two 
separate  steps.  First  the  participants  were  briefed  in  a  four  to 
five  minute  session.  This  was  a  straightforward  explanation  of 
the  purpose  of  the  survey,  why  they  were  selected  to  participate. 
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and  administrative  details.  The  second  step  was  to  give  the 
questionnaire  to  the  raters  and  request  they  read  the  instructions, 
complete  it,  and  return  it  before  the  expected  release  date  of  the 
senior  service  college  selectees  list  by  the  Department  of  the 
Army.  The  timing  allowed  five  days  for  the  raters  to  complete 
their  questionnaires.  Of  the  sixty- two  questionnaires  sent  out, 
fifty-seven  were  re.umed  completed. 

For  the  subordinate  ratings,  similar  procedures  were  followed 
at  Fort  Leavenworth.  In  that  group  235  of  the  247  questionnaires 
sent  out  were  returned. 

COMPILATION  OF  RESULTS 


All  returned  questionnaires  were  number  coded  for  identifica¬ 
tion.  The  data  from  the  questionnaires  were  punched  on  data  cards 
and  a  program  was  written  for  a  computer-*  to  extract  the  informa¬ 
tion  which  was  pertinent.  At  the  heart  of  this  step  was  the 
crucial  decision  of  converting  the  association  scale  (1-5)  and 
suitability  scale  (1-5)  into  some  meaningful  number^  in  order  that 
a  peer  order  of  merit  could  be  constructed.  At  Appendix  B  is  the 
numerical  value  associated  with  every  possible  combination  of  the 
association  and  suitability  scales.  On  this  scale,  the  higher 
numerical  value  indicates  the  individual  is  more  desirable  for 
selection;  conversely,  the  lower  value  indicates  a  lesser  desir¬ 
ability  for  selection.  Generally  the  rationale  is:  1)  Numerical 
values  in  the  1-5  range  are  definitely  negative  because  they 
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contain  the  "l"  suitability  rating.  Of  this  grouping,  an 
association  scale  of  "5"  (the  rater  who  best  knows  the  ratee)  is 
given  the  lowest  value  as  his  judgment  is  best  founded--i.e. ,  it 
is  the  most  negative  appraisal  available.  2)  Numerical  values  in 
the  6-10  range  were  similarly  derived.  These  ratings  are  also 
considered  to  be  on  the  negative  side  of  the  ledger.  Again  they 
are  arranged  so  that  an  association  scale  of  "1"  is  less  damaging 
to  a  ratee  than  is  an  association  scale  of  "5.”  3)  numerical 

values  in  the  11-15  range  are  considered  to  be  slightly  positive. 
Hence  the  association  scale  at  this  point  is  reversed  placing  the 
association  ratings  in  ascending  order  to  directly  reflect  increas¬ 
ing  numerical  value  equivalents  as  the  association  score  increases. 
4)  Finally  the  numerical  values  from  16-25  were  considered  as  a 
group.  It  is  here  we  find  the  most  positive  statements  about  a 
ratee.  While  this  portion  of  the  scale  could  be  constructed  some¬ 
what  differently  by  different  evaluators,  the  solution  presented 
has  validity  and  has  been  used.  This  scale,  1-25,  has  been  used 
to  place  the  ratees  in  a  peer  rank  order  of  merit. 

The  results  of  this  str’y  could  not  be  foreseen  with  any 
clarity.  For  this  reason  the  scoring  scale  outlined  in  the 
previous  paragraph  was  worked  out  in  detail  to  provide  a  very 
discriminate  rank  order  of  merit.  It  was  expected  this  would  be  a 
great  assist  In  mathematically  analyzing  the  results.  A  study  of 
the  results  indicated  this  could  not  be  done  very  meaningfully. 

For  these  reasons,  the  results  of  the  study  have  been  analyzed 
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with  a  layman's  approach  and  only  very  general  inferences  have 
been  drawn.  The  data  simply  did  not  materialize  as  definitively 
as  expected. 
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FOOTNOTES  FOR  CHAPTER  II 


1.  US  Departiijent  of  the  Army,  Senior  Service  College 
Selection  List  (April  1969). 

2 .  Malone  Interview. 

3.  Interview  with  Ronnie  L.  Morgan,  lLT,  USAWC  Staff  and 
Faculty,  Carlisle  Barracks,  Pennsylva  lia,  January  1971. 


4.  Ibid. 
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CHAPTER  III 


ANALYSIS  OF  RESULTS  FROM  PEER  RATINGS 

To  begin  t.he  analysis  wt  will  look  at  the  entire  spectrum  of 
the  peer  rating.  At  Appendix  C  is  a  chart  depicting  the  frequency 
of  response  by  the  raters  of  the  ratees.  This  is  surprisingly  low 
from  what  was  expected.  It  is  readily  seen  that  while  an  officer 
may  be  well  known  by  a  few,  he  is  not  necessarily  known  by  many 
(at  least  for  this  purpose  by  this  group  of  raters).  Or  looking 
at  it  from  a  different  angle,  each  peer  knew,  on  an  average,  less 
than  five  percent  of  the  population!  This  certainly  was  a  star¬ 
tlingly  low  number,  and  is  perlaps  one  of  the  most  significant 
observations  from  the  study. 

The  examples  of  peer  ratings  in  Part  I  all  dealt  with  small 
populations  of  groups  wfo  lived  in  fairly  closely  (physical 
proximity)  associated  life.  This  almost  dictated  they  would  get 
to  know  each  other  well,  or  at  least  become  somewhat  acquainted 
with  each  other.  The  particular  group  of  peers  in  this  study  does 
not  fall  into  that  neat  a  grouping.  They  do  not  live  in  any  sort 
of  closed  environment  with  each  other  and  their  acquaintanceship 
is  by  chance  and  is  intermittant  to  a  great  extent.  Additionally, 
the  acquaintance  has  spanned  anywhere  from  only  very  recently  to 
many,  many  years.  It  is  perfectly  possible  for  rater  ’"A"  to  have 
known  ratee  "B"  for  years  but  not  to  have  been  associated  directly 
with  him  for  years.  "A"  then  might  rate  "B"  without  any  hard 
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knowledge  of  his  actions,  deeds  and  service  in  the  intervening 
years  which  certainly  affect  his  selection  for  attendance  at  a 
senior  service  college.  The  particular  fact  demonstrated  here, 
the  low  apparent  acquaintanceship  of  these  peers,  will  be  very 
startling  to  many.  Personal  experience  and  observations  lead  many 
people  to  feel  otherwise.  This  observation  of  very  low  acquaintance¬ 
ship  is  not  conclusive,  nor  is  it  meant  to  indicate  it  would  be 
true  in  the  other  branches  of  the  Army.  But  it  is  true  of  this 
group  of  peers.  It  does  not  indicate  anything  about  the  size  of 
a  peer  group,  only  that  better  acquaintance  is  needed  to  get  a 
greater  number  of  responses.  The  degree  of  acquaintance  of  this 
group  of  peers  with  this  group  of  ratees  was  overestimated  at  the 
beginning  of  this  study.  Summarized,  the  following  factors  are 
deemed  pertinent  in  helping  to  explain  this  low  level 
of  acquaintance: 

1.  The  Infantry  branch  is  very  large  as  indicated  with 
this  list  of  nearly  2000  officers  eligible  for  attendance  at  a 
senior  service  college  next  year.  The  sheer  magnitude  of  numbers 
of  officers  makes  it  difficult  for  one  individual  to  know  a  great 
proportion  of  those  in  the  branch. 

2.  Specialty  programs  such  as  flying,  automatic  data 
processing,  special  forces,  research  and  development,  foreign  area 
specialist,  and  others  tend  to  compartmentalize  officers  to  a 
certain  extent,  and  hence  limit  their  exposure  to  certain  groups 
of  peers.  This  is  a  two  way  street  as  it  serves  as  an  obstacle 
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for  Che  specialist  to  become  acquainted  through  assignrents  with 
generalists  and  other  specialists,  and  also  a  hinderance  for  the 
generalist  to  become  better  acquainted  with  the  specialists. 

3.  The  most  recent  three  to  five  assignments  (as  a 
field  grade  officer)  of  a  USAWC  student  would  typically  not  bring 
him  in  contact  with  his  peers  (as  used  in  this  study)  nearly  as 
often  as  it  brings  him  in  contact  with  his  subordinates  and  his 
superiors.  (In  most  organizations  and  offices,  a  field  grade 
officer  has  few  peers,  but  many  subordinates  and  superiors.) 

During  the  overview  is  an  appropriate  time  to  mention  some  of 
the  shortcomings  of  the  survey.  First  of  these  is  the  fact  the 
original  list  of  over  7000  names  had  not  been  totally  "purified". 
There  were  very  few  names  on  it  which  were  there  incorrectly  and 
there  were  also  very  few  omissions.  Second,  in  making  a  pure 
Infantry  branch  list  from  the  original  list,  one  name  was  inad¬ 
vertantly  omitted.  Third,  three  of  the  names  on  the  questionnaire 
were  found  to  be  misspelled.  This  could  lead  to  erroneous  identi¬ 
fication  by  the  raters.  Fourth,  while  only  two  of  the  raters 
expressed  irritation  at  the  number  of  names  on  the  list,  and  at  the 
smallness  of  the  print,  there  were  probably  others  who  felt  the 
same.  This  could  result  in  receiving  invalid  responses  from  some 
raters.  The  fifth  inaccuracy  could  result  from  the  raters  confusing 
officers  with  like  last  names  and  similar  or  like  first  names  and 
middle  initials.  This  possible  shortcoming  is  very  difficult  to 
evaluate  and  it  cannot  be  eliminated.  Taken  altogether,  these 
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few  possible  shortcomings  should  not  have  an  appreciable  effect 
on  the  results  of  the  survey. 

BOARD  SELECTION 


After  this  general  overview  of  the  entire  list  of  responses 
we  will  analyze  specific  portions  of  the  responses.  First  to  look 
at  the  ninety-five  names  which  were  selected  by  the  selection 
board.  These  ninety-five  names  "onsist  of  eighty  who  are  primary 
selectees  and  fifteen  alternates.  The  selection  board  placed  them 
in  a  rank  order  of  merit  1-95.  The  table  at  Appendix  D  indicates 
the  frequency  of  response  by  the  peers  for  these  ninety-five 
selectees  as  well  as  indicating  the  number  of  officers  the  board 
and  the  peers  agreed  upon.  The  most  startling  observation  here  is 
that  five  of  these  top  flight  infantry  officers  were  not  known  by 
a  single  peer!  Further  over  half  of  the  ninety-five  were  known  by 
fewer  than  five  of  their  peers.  In  other  words,  over  half  of  the 
cream  of  the  officers  are  virtually  unknown  by  their  peers.  It 
seems  hardly  conceivable  that  if  four  or  fewer  peers  were  to 
recommend  to  a  board  that  they  select  or  not  select  an  officer 
for  attendance  at  a  senior  service  college  that  the  board  would 
consider  the  recommendation.  If  this  concept  is  accepted,  our 
value  placed  on  a  sort  of  peer  rating  described  here  is  of 
doubtful  value.  Opposed  to  this  peer  system  where  so  few  ratees 
are  known  by  so  few  raters,  is  the  in-being  system  where  each 
ratee's  record  is  meticulously  screened  by  many  senior  officers. 
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Under  that  system,  each  ratee  is  assured  maximum  exposure  and 
consideration.  Referring  to  column  three  of  Appendix  D  of  these 
top  ninety-five,  the  peers  agreed  and  selected  fourteen.  Again 
we  see  that  eight  of  these  fourteen  ratings  are  based  on  only  one 
or  two  ratings  each — hardly  suitable  as  a  criterion  for  a  selection 
of  this  importance.  It  must  be  borne  in  mind  that  the  selection 
boar*  was  considering  only  259  names  where  the  peers  were  consider¬ 
ing  nearly  2000.  Later  we'll  see  a  more  direct  comparison  using 
only  the  259  names. 


PEER  SELECTION 


Next  under  consideration  is  the  top  ninety- five  in  the  peer 
rank  order  of  merit  at  Appendix  E.  First,  in  doing  this,  no 
cognizance  was  taken  of  the  number  of  responses  to  determine  the 
peer  rank  order  of  merit.  Sixty-four  officers  were  selected  in  the 
top  ninety-five  based  on  only  one  response  each!  This  selection 
based  on  only  one  response  trould  be  manifestly  unwise.  However,  the 
one  officer  who  garnered  twelve  responses  was  also  selected  by  the 
board.  His  peerc  put  him  in  rank  order  sixty  (behind  many  of 
those  with  only  one  or  two  responses)  while  the  selection  board 
put  him  in  rank  order  thirty.  Continuing  in  this  vein,  those  two 
officers  who  received  eight  and  six  responses  were  in  rank  order 
eighty- four  and  fifty-six  respectively  on  the  peer  scale  comrared 
to  rank  order  eight  and  thirty-one  respectively  on  the  selection 
board's  rank  order.  The  one  officer  who  received  seven  ratings 
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was  not  selected  by  the  selection  board,  although  his  name  was 
considered  by  them.  With  this  meagre  sample,  five  out  of  six  of 
the  top  ninety-five  who  received  five  or  more  ratings  were  selected 
by  both  the  selection  board  and  by  their  peers. 

RATINGS  OF  SELECT  POPULATION 


Now  we  turn  to  consider  how  the  peers  rated  only  the  259  names 
which  the  selection  board  considered.  Recall  these  259  names  were 
supplied  to  the  selection  board  by  the  0P0  from  the  total  of  nearly 
2000  eligibles.  Recall  also  that  these  were  essentially  peers  who 
screened  the  2000  records  to  select  these  259  names.  At  Appendix 
F  is  the  comparison  of  the  selection  board's  top  ninety- five  with 
the  peer  ranking  of  this  group  as  well  as  an  indication  of  how  the 
board  and  the  peer  group  agreed.  With  this  more  select  group, 
there  is  a  much  higher  number  of  selectees  selected  by  the  peers 
who  were  also  selected  by  the  board,  than  with  the  entire  popula¬ 
tion  as  indicated  at  Appendix  D.  A  result  of  this  nature  is  to  be 
expected  as  the  population  from  which  the  selectees  is  diminished, 
greater  agreement  will  be  reached  unless  the  selectees  are  unknown 
to  the  raters,  A  low  level  of  response  is,  however,  still  present. 
Over  half  of  the  selectees  had  only  five  or  fewer  ratings.  Even 
with  this  more  select  group  a  low  level  of  acquaintance  is 
still  displayed. 

To  look  at  this  in  a  different  manner,  turn  now  to  Appendix  G. 
Even  with  this  select  group  of  officers,  well  over  half  were 
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selected  in  t.  a  top  ninety- five  by  only  four  or  fewer  raters. 

Looking  only  at  those  officers  vho  received  eight  or  more  ratings, 
the  peers  agreed  quite  well  with  the  board.  In  this  grouping  the 
board  selected  twenty-three  officers  and  the  peer  ratings  agreed 
with  fourteen  of  them,  resulting  in  nearly  sixty  percent  agreement, 
in  this  entire  survey,  this  is  the  greatest  agreement  to  be  found 
between  the  peer  ratings  and  the  selection  board  selections. 

Granted,  it  only  represents  a  small  segment  of  this  study,  but  it 
is  the  one  which  seems  to  indicate  the  greatest  validity  of  these 
peer  ratings  (assuming  the  board  selection  is  correct)!  Remember¬ 
ing  that  to  get  this  agreement  it  was  necessary  first  to  limit  the 
rated  population  (to  259)  and  further  to  discard  the  population 
rated  fewer  than  eight  times,  this  might  give  a  better  start  point 
for  any  future  work  with  this  sort  of  peer  group  rating  study. 

THE  BOTTOM  NINETY-FIVE 

The  final  analysis  to  be  made  regarding  the  peer  rating 
portion  of  this  study  is  outlined  at  Appendix  H.  Here  is  displayed, 
again  using  only  the  limited  population  of  259,  the  bottom  ninety- 
five  in  peer  rank  order  of  merit.  Xmnediately  noticeable  is  the 
few  number  of  times  officers  are  rated.  Over  two-thirds  of  them 
were  rated  by  three  or  fewer  peer  .  The  same  five  officers  who 
were  unknowr  on  Appendix  D  show  up  unknown  here  so  they  shall  not 
be  discussed  further.  Dropping  to  the  lower  portion  of  Appendix 
H,  to  the  officers  who  received  seven  or  more  responses,  there  is 


considerable  disagreement  with  the  selection  board.  Here  are  ten 
officers  who  were  rated  definitely  low  by  their  peers,  yet  the 
selection  board  selected  four  of  them  in  the  top  ninety-five.  An 
indication  of  this  nature  could  lend  some  weight  to  the  delibera¬ 
tions  of  a  selection  board.  Again,  as  was  pointed  out  in  the 
explanation  of  Appendix  C,  this  would  have  to  be  studied  further 
with  greater  definition. 

Here  we  end  the  analysis  of  the  peer  ratings  as  compared  with 
the  selection  board.  None  of  them  is  overly  revealing,  but  they 
do  point  to  areas  where  further  work  could  be  done.  One  of  these 
prime  areas  is  ensuring  the  raters  are  well  familiar  with  the 
individuals  they  are  to  rate.  This  is  of  all  things  a  hard  and 
fast  prerequisite  for  a  peer  rating  system.  Another  area  which 
needs  further  refinement  is  in  the  length  of  list  of  names  of 
individuals  to  be  rated.  The  conclusions  and  rec-oimiendations  in 
Chapter  V  emphasize  these  points. 
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FOOTNOTES  FOR  CHAPTER  III 


1.  Interview  with  Francis  W.  Craig,  LTC,  Office  of  the 
Deputy  Chief  of  Staff  for  Personnel,  US  Ariny,  Washington,  D.  C., 
1970. 
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CHAPTER  IV 


ANALYSIS  OF  RESULTS  FROM  SUBORDINATE  RATINGS 

1  uming  now  from  the  peers*  to  the  subordinate® '  ratings. 

In  this  analysis  it  is  again  demonstrated  as  it  was  in  the  peer 
ratings,  there  is  a  general  lack  of  acquaintance  with  the  ratees 
and  a  low  level  of  response.  The  same  start  point  for  this 
analysis  is  used  as  in  the  case  of  the  peers.  Appendix  C. 

Referring  back  to  Appendix  C,  it  reveals  about  the  same 
distribution  of  the  number  of  responses  from  the  subordinates  as 
we  had  for  ..-.e  peers.  They  do  cover  a  broader  spectrum  (to  a  high 
of  forty-seven  responses)  but  there  are  very  few  in  proportion 
below  the  fifteen  response  level.  On  an  average  each  subordinate 
rated  less  than  two  percent  of  the  population! 

The  same  general  comments  made  for  the  peers  relative 
acquaintance  and  recentness  of  association  with  the  ratees  can  be 
made  for  the  subordinates.  This  parameter  has  not  been  rigidly 
defined  beyond  the  association  scale  explained  on  the  questionnaire. 

Now  to  look  at  the  results  of  the  subordinate  ratings.  They 
will  be  presented  in  the  same  order  as  those  for  the  peers,  but 
with  a  slightly  briefer  explanation. 

BOARD  SELECTION 

Appendix  I  relates  the  ninety- five  officers  selected  by  the 
board  to  the  number  of  responses  on  this  population  and  shows  a 
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very  low  level  of  agreement  with  the  subordinates'  selection. 

The  subordinates  in  this  case  only  selected  four  of  the  same 
officers  as  did  the  board.  These  four  were  selected  based  on 
only  one  rating  each!  There  is  little  positive  value  in  this 
relationship.  It  does  tend  to  highlight  again  the  low  level  of 
acquaintance  the  subordinates  have  witn  this  population. 

SUBORDINATE  SELECTION 

Appendix  J  for  the  subordinates  is  the  companion  to  Appendix 
E  for  the  peers.  Where  the  peers  selected  some  of  the  population 
to  be  in  the  top  ninety- five  based  on  several  ratings,  the  sub¬ 
ordinates  have  selected  all  of  theirs  on  only  one,  two,  or  three 
ratings  each.  As  is  indicated,  most  of  these  were  selected  based 
on  only  one  rating.  Apparently  it  is  a  case  of  one  subordinate 
knowing  one  of  the  ratees  well  and  giving  him  a  maximum  score. 
Actually  on  the  computer  print  out,  these  seventy- four  who  were 
rated  once  each,  were  all  tied  for  first  .place  in  the  rank  order 
of  merit.  To  get  on  to  something  a  little  more  meaningful,  refer 
now  to  Appendix  K. 

RATINGS  OF  SELECT  POPULATION 

It  becomes  a  little  more  definitive  as  under  consideration 
here  is  only  the  population  of  259  names  the  selection  board 
actually  considered.  While  the  subordinates  are  in  much  greater 
agreement  with  the  board  here  than  when  the  entire  population  was 
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considered  (Appendix  J),  still  over  half  of  the  selectees  were 
selected  based  on  three  or  fewer  ratings.  Note  should  be  taken 
of  those  selectees  who  were  rated  by  fifteen  and  nineteen  raters 
and  still  were  selected  in  the  top  ninety-five.  No  real  signifi¬ 
cance  can  be  placed  on  them,  however,  it  might  indicate  a  higher 
threshold  of  number  of  ratings  for  selection  is  needed.  The 
threshold  used  in  this  study  was  one. 

Appendix  L  depicts  the  comparison  between  the  top  ninety-five 
selected  by  the  subordinates  with  the  ranking  of  that  ninety-five 
bv  the  selection  beard.  Only  the  259  names  considered  by  the 
selection  board  are  considered  here.  There  is  a  remarkable  amount 
of  agreement  in  this  comparison.  This  is  the  same  comparison  in 
which  the  peer  ratings  looked  so  favorable.  Using  only  eight  or 
more  responses  as  the  threshold  of  interest,  the  chart  at  Appendix 
L  reveals  sixty-six  percent  agreement  between  the  subordinate 
ratings  and  the  selection  board  se.ections!  Sixty-six  percent  is 
even  slightly  higher  than  the  pee-.s  had  on  this  particular  com¬ 
parison  at  Appendix  G.  Again  at  Appendix  L  we  see  the  familiar 
great  number  of  selectees  based  on  three  or  fewer  ratings. 

THE  BOTTOM  NINETY -FIVE 

Appendix  M  gives  the  basis  for  some  disagreement  between  the 
subordinates  rating  and  the  selection  board's  selection.  Four 
cases  in  point  stand  out;  those  four  wnich  received  more  than  ten 
ratings  each.  The  subordinates  were  relatively  convinced  these 
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belong  in  the  bottom  ninety-five  while  the  selection  board  placed 
them  in  rank  order  nine,  fifty-four,  nir*c;y-one  and  ninety-three 
from  the  122  of  their  list.  Something  in  this  order  of  magnitude 
could  be  considered  by  a  selection  board.  Perhaps  it  would  cause 
the  board  to  take  a  second  look  at  the  individuals'  records  and 
compare  them  more  critically  with  others  being  considered,  pro¬ 
viding  the  others  also  had  some  meaningful  indication  from  a 
subordinate  rating. 

PEER  RATINGS  VS  SUBORDINATE  RATINGS 

The  next  comparison  made  in  this  study  was  between  the 
selection  of  the  top  ninety-fiv“  officers  as  selected  by  the  peers 
and  the  corresponding  rating  of  these  same  ninety-five  by  the 
subordinates.  That  comparison  is  shown  at  Appendix  N.  Here  is 
the  same  top  ninety- five  portrayed  in  Appendix  E  and  it  shows 
that  the  subordinates  agreed  with  twelve  of  these  selections.  The 
twelve  agreements  are  only  based  on  one  or  two  ratings  however. 

The  final  comparison  in  this  study  is  at  Appendix  0.  For  the 
purpose  of  this  comparison,  the  entire  population  was  considered. 
Only  those  ratees  who  had  at  least  eight  ratings  by  both  their 
peers  and  their  subordinates  were  used.  Of  these  thirty,  thirteen 
had  been  considered  by  the  board  for  selection  at  a  senior  service 
college  and  eight  were  actually  selected  in  the  top  ninety-five. 

Of  in' jrest  is  the  fact  that  neither  the  peers  nor  the  subordinates 
selected  any  of  the  top  ninety-five  which  the  board  selected! 
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Further,  the  only  ratee  on  this  list  who  was  in  the  top  ninety- 
five  in  either  the  peer  or  subordinate  order  of  merit,  was  in  the 
top  ninety- five  on  both  lists,  was  the  most  rated  individual  by 
the  subordinates  and  the  second  most  rated  by  the  peers,  and  was 
not  even  considered  by  the  board  in  its  select  list  of  259!  A 
great  varii-nce  in  order  of  merit  becween  the  peers  and  subordinates 
is  evident.  The  difference  in  rank  o&der  varies  from  a  low  of 
only  three  to  a  high  of  565  with  the  average  difference  in  ranks 
being  slightly  over  278.  The  two  orders  of  merit  show  very  little 
similarity  to  each  other. 

The  comparison  at  Appendix  0  is  meaningful  to  the  extent  it 
is  the  only  comparison  between  peer  and  subordinate  ratings  made 
from  the  greater  number  of  responses  and  indicates  very  little 
agreement  among  the  ratings  of  the  peers,  selection  board  and 
subordinates.  The  number  of  responses  in  this  group  is  still  very 
low  to  use  as  a  basis  for  valid  peer  or  subordinate  ratings,  but 
it  has  the  greatest  number  of  responses  of  any  of  the  ratings  in 
this  study. 
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CHAPTER  V 


CONCLUSIONS  AND  RECOJ*4ENDATIONS 

Unfortunately,  no  positive  conclusions  pertaining  to  the 
value  of  peer  or  subordinate  ratings  can  be  drawn  from  this  study. 
At  the  outset,  it  was  hoped  to  derive  positive  insight  into  tke 
value  of  these  ratings  in  this  real  life  situation.  There  are 
some  observations  which  can  be  made  as  a  result  of  the  data  which 
were  collected,  although  they  do  not  directly  bear  on  the  question 
of  the  value  of  peer  ratings. 

The  first  of  these  has  to  do  with  how  well  peers  and  subor¬ 
dinates  are  acquainted  with  each  other.  In  both  cases,  peers  and 
subordinates,  this  study  revealed  an  insufficient  number  of  raters 
knew  an  insufficient  number  of  ratees  to  amass  a  meam  gful  number 
of  responses  fo*-  a  rating  system.  The  magnitude  of  the  response 
proved  to  be  considerably  less  than  desired.  That  is,  a.  good 
proportion  of  the  infantry  olficers  answered  the  questionnaire,  but 
they  indicated  a  very  low  level  of  acquaintance  with  the  ratees. 

In  fact,  this  low  number  of  responses  per  ratee  has  cast  the 
expected  outcome  of  the  study  in  a  different  light  than  expected. 
The  results  are  more  a  measure  of  how  well  this  group  of  peers 
knows  the  » itees  than  it  is  a  good  measure  of  how  peer  ratings 
could  assist  a  selection  board.  One  does  not  fully  realize  the 
hundreds  of  people  in  his  own  profession  he  does  not  know  until  he 


is  given  a  roster  ''f  them,  such  as  in  this  survey,  and  asked  to 
identify  those  few  he  does  know. 

Secondly,  there  generally  appeared  to  be  very  little  rela¬ 
tionship  between  the  rank  order  of  .urit  constructed  by  the  peers 
versus  that  of  the  subordinates.  This  could  be  caused  by  a  great 
variety  of  factors  from  an  honest  difference  of  opinion,  to  pro¬ 
fessional  jealousy,  to  a  different  perspective,  to  a  difference  in 
a  basis  of  acquaintance,  to  a  difference  in  maturity  on  the  part 
of  the  rater,  to  mention  a  few. 

Finally,  we  conclude  there  appeared  tc  be  very  liccle  basis 
for  comparing  the  peer  rank  order  of  merit  and  the  selection 
board's  selections  and  also  comparing  the  subordinate  rank  order 
of  merit  and  the  selection  board's  selections.  Of  course  this  vas 
one  of  the  m^jor  reasons  for  conducting  the  study,  but  data  in 
sufficient  quantity  just  were  not  forthcoming. 

Based  on  the  experience  of  this  study  and  the  results  from 
it,  the  following  recommendations  are  made: 

1.  The  data  which  have  been  collected  should  be  further 
studied  and  manipulated  for  additional  possible  benefit.  Partic¬ 
ular  attention  should  be  given  to  extract  results  based  on  various 
thresholds  of  numbers  and  responses. 

2.  A  simila*  study  be  undertaken  next  year  using  the 
corresponding  selection  board.  The  difference  being  that  only 
the  list  the  selection  board  uses  should  be  subject  to  peer  and 
subordinate  ratings,  restricting  the  list  of  names  to  only  the 
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ones  the  selection  board  uses  is  a  p»-ime  consideration.  It  not 
only  makes  the  ratee's  task  much  easier  but  also  eliminates  a  great 
deal  of  chaff  for  the  person  analyzing  the  results.  A  similar 
study  should  place  thresholds  on  the  recentness  of  acquaintance 
of  the  rater  with  the  ratee.  Such  a  system  would  serve  to  further 
refine  the  results  and  provide  a  truer  indication  of  the  value  of 
the  rating  by  the  peers  and  subordinates.  If  an  approach  of  this 
nature  were  followed,  it  would  be  very  feasible  and  desirable  to 
conduct  the  study  using  all  branches  of  the  Army  represented  on 
the  list. 


78 


BIBLIOGRAPHY 


1.  Adams,  Donald  K.,  et.  al.  Assessment  of  Men.  New  York: 

Rinehart  &  Company,  Inc.,  1948.  (BF341U6) 

2.  Adams,  Joe  Kennedy.  Basic  Statistical  Concepts.  New  York: 

McGraw  Hill  Book  Company,  Inc.,  1955.  (HA29  A35) 

3.  Adler,  Irving.  Probability  and  Statistics  for  Every  Man:  How 

to  Understand  and  Use  the  Laws  of  Chance.  New  York:  New 
American  Library,  1966.  (A273) 

4.  Craig,  Francis  W. ,  LTC.  Office  of  the  Deputy  Chief  of  Staff 

for  Personnel,  US  Army,  Washington,  D.  C.:  Personal  Interview, 
November  1969. 

5.  Creuziger,  Donald  P.  The  Problems  and  Relationships  of  Pro¬ 

motion  and  Military  Education.  US  Army  War  College,  Carlisle 
Barracks,  Pennsylvania,  1969.  (AWC  IS  69) 

6.  Croxton,  Frederick  Emory,  and  Cowden,  Dudley  J.  Applied  General 

Statistics.  New  York:  Prentice-Hall,  Inc.,  1940.  (HA29  C776) 

7.  deJung,  John  E.,  and  Kaplan,  Harry.  "Some  Differential  Effects 

of  Race  of  Rater  and  Ratee  on  Early  Peer  Ratings  of  Combat 
Aptitude.1'  Journal  of  Applied  Psychology,  Vol.  46,  1962, 
pp.  370-374. 

8.  DF,  Subject:  Experimental  Peer  Rating  System,  Director  of 

Resident  Instruction,  CGSC,  Fort  Leavenworth,  Kansas, 

11  July  1968. 

9.  Dooher,  M.  Joseph,  and  Marquis,  Vivienne,  ed.  Rating  Employee 

and  Supervisory  Performance.  New  York:  American  Management 
Association,  1954.  (HF5559D6) 

The  following  articles  were  used  from  this  book:  "Merit 
Rating:  Its  Validity  and  Techniques,"  by  Joseph  Tiffin, 
pp.  11-19;  "Developing  an  Employee  Merit  Rating  Procedure," 
bv  Reign  Bittner,  pp.  20-34;  "Planning  for  a  Successful 
Merit  Rating  Program,"  by  Gustav  C.  Hertz,  pp.  43-46; 

"Let's  Get  More  Scientific  in  Rating  Employees,"  by 
Walter  R.  Mahler,  pp.  49-59;  "A  Merit  Rating  Procedure 
Developed  by  and  for  the  Raters,"  by  Edward  A.  Rundquist 
and  Reign  H.  Bittner,  pp.  69-79. 

10.  Franz biau,  Abraham  Norman.  A  Primer  of  Statistics  for  Non- 

Statisticians;  New  York:  Harcourt,  Brace,  1958.  (HA29  F65) 


79 


11. 


Guilford.  J.  P.  Psychometric  Methods.  New  York  and  London: 
McGraw  Hill  Book  Company,  Inc.,  1936. 

12.  Hoel,  Paul  G.  Elementary  Statistics.  New  York,  London, 

Sydney:  John  Wiley  and  Sons,  Inc.,  1966.  (HA29  H62) 

13.  Hollander,  E.  P.  "The  Reliability  of  Peer  Nominations  Under 

Various  Conditions  of  Administration."  Journal  of  Applied 
Psychology,  Vol.  41,  No.  2,  1957,  pp.  85-90. 

14.  _ .  "Leadership,  Followership,  and  Friendship:  An 

Analysis  of  Peer  Nominations."  The  Journal  of  Abnormal 
and  Social  Psychology,  Vol.  50,  1955,  pp.  163-167. 

15.  Keller. ,  Konard.  "On  Problems  in  Perceiving  Other  Nations  and 

Systems."  Security  Studies  Project  Number  15.  Los  Angeles, 
California:  University  of  California,  1968. 

16.  Kubany,  Albert  J.  "Use  of  Sociomntric  Peer  Nominations  in 

Medical  Education  Research."  Journal  of  Applied  Psychology, 
Vol.  41,  No.  6,  December  1957,  pp.  389-394. 

17.  Malone,  Dandridge  M. ,  LTC.  Staff  and  Faculty  of  the  US  Army 

War  College,  Carlisle  Barracks,  Pennsylvania:  Personal 
Interviews  October  1970  through  February  1971 

18.  Mandell,  Milton.  "Appraisals:  A  Valid  Management  Selection 

Tool?"  Personnel,  Vol.  35,  No.  3,  November-December  1968. 

19.  Maurer,  William  W. ,  LTC.  Student  at  the  US  Army  War  College, 

Carlisle  Barracks,  Pennsylvania:  Personal  Interviews 
December  1970  through  January  1971. 

20.  Morgan,  Ronnie  L.,  1LT  Staff  and  Faculty  of  the  US  Army 

War  College,  Carlisle  Barracks,  Pennsylvania:  Personal 
Interviews  October  1970  through  February  1971. 

21.  Moroney,  M.  J.  Facts  from  Figures.  Baltimore:  Penguin  Books, 

1967.  (HA29  M68) 

22.  Mosel,  James  N.  "Human  Organization  and  Effective  Training." 

Address.  Training  Officers  Conference,  Washington,  D.  C., 

14  February  1956,  reprinted  in  Recommended  Reading  Personnel 
Management  for  Executives,  European  Region,  DA,  n.d.,  pp.  1-11. 

23.  Nadal,  Ramon  A.,  MAJ.  "A  Review  of  Peer  Rating  Studies." 

Research  Report  No:  68-8.  West  Point,  New  York,  October  1968. 


80 


24.  Norris,  Awbrey  G.,  COL.  Student  at  the  US  Army  War  College, 

Carlisle  Barracks,  Pennsylvania:  Personal  Interviews 
December  1970  through  February  1971. 

25.  OERS  Study  Group,  ODCSPER-CSD,  Office  Deputy  Chief  of  Staff, 

Personnel,  Department  of  the  Army.  The  Officer  Efficiency 
Reporting  System,  OERS,  FOR  OFFICIAL  USE  ONLY,  n.p., 

25  June  1969. 

26.  Rehm,  Thomas  A.,  LTC.  "Ethics  and  the  Military  Establishment." 

Military  Review,  Vol.  L,  No.  9,  Fort  Leavenworth,  Kansas, 
September  1970,  pp.  9-14. 

27.  Roadman,  Harry  E.  "An  Industrial  Use  of  Peer  Ratings." 

Journal  of  Applied  Psychology,  Vol.  48,  1964,  pp.  311-314. 

28.  Rowland,  Virgil  K.  "The  Mechanics  of  Group  Appraisal." 

Personnel ,  Vol.  34,  No.  b,  June  1968. 

29.  The  Officer  Biographical  Sketches,  1971,  US  Army  War  College, 

Carlisle  Barracks,  Pennsylvania,  1970. 

30.  Tupes,  E.  C.  Relationships  Between  Behavior  Trait  Ratings  by 

Peers  and  Later  Officer  Performance  of  USAF  Officer  Candi¬ 
date  School  Graduates.  Air  Force  Personnel  and  Training 


31.  U 


32.  US  Department  of  the  Artuy.  Circular  No;  351-4.  1969  Senior 

Service  College  Selection  System.  Washington,  D.  C., 

31  October  1969. 

33.  US  Department  of  the  Army.  Circular  No:  351-18.  Senior 

Service  College  Selection  System.  Washington,  D.  C., 

29  October  1970. 

34.  US  Department  of  the  Army.  Message  DA  3151811.  Senior 

Service  College  Level  Schooling.  Washington,  D.  C., 

11  November  1970. 

35.  US  Department  of  the  Army.  Senior  Service  College  Sele-tion 

List.  April  1969. 

36.  US  Department  of  Defense.  Assistant  Secretary  of  Defense  for 

Manpower.  Officer  Education  Studv.  Washington,  D.  C., 

July  1966.  (U4C8.3  A38) 


Center:  Lackland  Mr  Force  P.ase,  October  1957. 


ilted  Nations.  Statistical  Office.  Handbook/on  Data 
Processing  Methods.  Rome,  1959.  (HA33  U4)_ 


81 


A 


APPENDIX  A 


1  D95POSITION  FORM  s:  6  Nov  70 

ttfUCNCI  oa  OffKX  SVMSOi 

AWCIR 

Survey 

TO  Each  Infantry  Student  COL  D.  H.  Wardrop  29  Oct  7U 

Student  Detachment 

US  Aray  War  College 

1.  The  attached  survey  is  part  of  a  student  research  project. 

CMT  « 

2.  Please  complete  the  survey;  enter  your  student  box  nuaber  In  the  space  provided; 
and  return  through  Message  Center  NLT  Friday,  6  Sioveaber. 


3.  Should  you  have  any  questions  regarding  the  survey,  feel  free  to  contact  ae 
at  USAWC  extension  4607  or  4482. 


4.  Your  cooperation  in  this  project  is  greatly  appreciated. 
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PLEASE  ENTER  YOUR  MESSAGE  CENTER:  RETURN  TO 

STUDENT  BOX  NO.  COL  D.  H.  WARDROP 

AND  CRADE:  0-  Box  191,  USAWC 


SURVEY :  SELECTION  FOR  SENIOR  SERVICE  COLLEGE 


The  objective  of  this  survey  is  to  get  on  Indication  of  the  possible  use 
of  peer  ratings  for  assisting  in  the  selection  of  Army  officers  for 
attendance  at  a  senior  service  college.  For  the  purpose  of  this  survey 
you  are  considered  a  peer  of  every  officer  on  the  enclosed  roster. 

The  enclosed  roster  eon  to  ins  nil  infantry  officers  In  the  xoue  of  ellgl- 
blllty  to  be  selected  to  attend  one  of  the  senior  service  colleges  during  ^ 
the  academic  year  1971-1972.  Request  you  perform  the  following  three 
actions  regarding  this  roster: 

1.  As  step  #1:  Underline  each  name  that  you  are  reasonably  sure  you 
recognize.  Then  perform  the  next  two  actions  jointly  as  step  #2. 

2.  ASSOCIATION:  There  are  Innumerable  ways  one  person  con  "know'* 
another.  The  scale  below  does  not  Include  all  possible  variations,  but 
rather  its  purpose  is  to  provide  an  indication  of  how  well  you  know  the 
individual  concerned.  From  the  scale,  please  select  the  choice  which 
best  represents  your  association  with  the  individual  whose  names  you  havs 
underlined.  Do  not  be  overly  concerned  with  how  long  tb*  relationship 
has  existed.  (For  each  name  underlined,  enter  appropriate  number  (1-5) 
in  "Association"  column.) 

I  know  individual  by: 

1.  Reputation  and/or  1-2  personal  contacts. 

2.  Minimum  social  and/or  professional  contact. 

3.  Occasional  social  and/or  professional  contact. 

A.  Frequent  social  and/or  professional  contact. 

5.  Close  and  frequent  social  and/or  professional  contact, 

3.  SUITABILITY:  In  this  operation,  please  indicate  your  opinion  of 
the  individual's  suitability  for  attendance  at  a  senior  service  college. 

Do  not  consider  hew  well  you  know  the  person— this  will  be  accounted  for 
in  the  association  scale  above.  As  a  general  criterion,  consider  thet 
those  who  attend  the  senior  service  colleges  will  probably  stove  upward 
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in  the  Army  to  position*  of  greater  responsibility.  (For  esch  name  under¬ 
lined,  enter  appropriate  number  (1-5)  in  "Suitability"  column*) 

In  my  opinion,  this  individual  should  bet 

1.  Not  selected. 

2.  Selected  later  if  warranted  by  record*. 

3.  Selected  next  year. 

A.  Selected  as  an  alternate  or  standby  this  year. 

5.  Selected  iawdiately  * 


Please  use  the  following  guidance  in  completing  your  roster: 

1.  There  is  no  limit  on  the  number  of  officers  you  select  for  any 
year,  or  rejects 

2.  There  an  w  "trick*"  or  bidden  meanings  in  this  survey.  Complete 
the  roarer  in  a  Judicl  w,  straightforward  manner. 

J.  If  you  are  aware  of  any  personal  desires  of  the  officer  under  con¬ 
sideration,  do  cot  take  these  into  account.  Make  a  selection  or  rejection 
bases  on  the  officer's  professional  qualification*  and  potential  aa  you 
know  them. 

I  nave  requester  your  box  number  am  the  first  page  ?f  the  survey  for  ray 
use  Sn  cam?  of  most iou*  after  you  return  it  to  rae.  I  shall  treat  your 
cuatpletcd  rcMfMnme  as  confidential  and  request  you  also  treat  it  in  s 
c«sa['idtfniiol  corner.  Please  do  not  discuss  your  response  with  other 
HtmuencH.  Ah  information  from  your  response  is  transferred  to  data  cards, 
yuur  idonLlfj  Lsaubup  with  the  roster  will  cease,  and  the  names  of  tha 
individuals  on  earn  roster  will  be  dropped  and  will  be  represented  by  a 
code  number  in  all  scoring,  analyses ,  and  presentations  of  results. 

Thank  you. 


1  End 

as 
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APPENDIX  B 


DERIVATION  OF  NUMERICAL  VALUES 
FOR  RESPECTIVE  ASSOCIATION  AND  SUITABILITY  SCORES 


Association 

Score 

5 

4 

3 
2 
1 

5 

4 
3 
2 
1 
1 
2 

3 

4 

5 
1 
2 
1 

3 
2 

4 

3 

5 

4 

5 


Suitability 

Score 

1 

1 

1 

1 

1 

2 

2 

2 

2 

2 

3 

3 

3 

3 

3 

4 

4 

5 

4 

5 

4 

5 

4 

5 
5 


Numerical 

Score 

1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 
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APPENDIX  C 


FREQUENCY  OF  RESPONSES  FROM  PEERS  AND  SUBORDINATES 


Number  of  Ratees  USAWC 

Number  of 

Humber  of  Ratees  C&GSC 

(Peers)  Responded  to: 

Responses: 

(Subordinates)  Responded  to: 

Column  1 

Column  2 

Column  3 

386 

0 

311 

476 

1 

387 

348 

2 

306 

244 

3 

239 

164 

4 

185 

105 

5 

129 

67 

6 

91 

45 

7 

71 

38 

8 

54 

32 

9 

42 

26 

10 

25 

17 

11 

28 

10 

12 

19 

4 

13 

15 

4 

14 

14 

7 

15 

11 

- 

16 

2 

1 

17 

7 

- 

18 

6 

- 

19 

4 

1 

20 

5 

- 

21 

2 

1 

22 

2 

- 

23 

3 

- 

24 

4 

- 

25 

2 

- 

26 

2 

- 

27 

2 

•» 

28 

3 

- 

29 

1  , 

- 

32 

1 

- 

35 

2 

- 

47 

1 

1976 

TOTAL 

1976 

5233 

Total  Responses 

7500 

(Col  1  X  Col  2)  for  AWC 
(Col  3  X  Col  2)  for  C&GSC 

Averse  No.  of 

92  Responses  per  Ratee:  32 
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APPENDIX  D 


FREQUENCY  OF  RESPONSE  BY  PEERS 
OF  OFFICERS  SELECTED  BY  THE  BOARD 


Column  one  indicates  the  number  of  responses  by  peers  on  the 
top  ninety- five  of  column  two. 

The  ninety- five  in  column  two  are  the  top  ninety-five 
selected  by  the  board.  The  number  of  ratees  indicated  in  column 
two  are  grouped  according  to  the  number  of  responses  they 
received,  as  indicated  in  column  one. 

The  number  of  ratees  indicated  in  column  three  are  grouped 
according  to  the  number  of  responses  they  received,  as  indicated 
in  column  one.  These  ratees  were  also  (along  with  the  respective 
ones  i-’  column  two)  in  the  top  ninety- five  as  determined  by 
the  peers. 


Column  1 

0 

1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 
20 


Column  2 

5 

11 

15 

12 

6 
6 
7 
6 
9 
5 
4 
4 
2 
1 
1 
1 


TOTAL  95 


Column  3 


3 

5 

«• 

1 

2 

1 

1 


1 


14 


Percent  agreement  is  14 
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APPENDIX  E 


FREQUENCY  OF  RESPONSE  BY  PEERS 
OF  TOP  NINETY-FIVE  SELECTED  BY  PEERS 


Column  one  indicates  the  number  of  responses  by  peers  on  the 
top  ninety- five  of  column  two. 

The  ninety- five  in  column  two  are  the  top  ninety-five  selected 
by  the  peers.  The  number  of  ratees  indicated  in  column  two  are 
grouped  according  to  the  number  of  responses  they  received,  as 
indicated  in  column  one. 

The  number  of  ratees  indicated  in  column  three  are  grouped 
according  to  the  number  of  responses  they  received,  as  indicated 
in  column  one.  These  .atees  were  also  (along  with  the  respective 
ones  in  column  two)  in  the  top  ninety- five  as  determined  by  the 
selection  board. 


Column  1 

1 

2 

3 

4 

5 

6 

7 

8 
12 


Column  2 

64 

18 

5 

2 

2 

1 

1 

1 

1 


Column  3 

3 

5 

1 

2 

1 


1 


TOTAL  95  14 


Percent  agreement  is  14 
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APPENDIX  F 


FREQUENCY  OF  RESPONSE  BY  PEERS  OF  OFFICERS  SELECTED  BY  THE  BOARD 
BASED  ONLY  ON  259  NAMES  SUPPLIED  TO  THE  BOARD 


Column  one  indicates  the  number  of  responses  by  peers  on  the 
top  ninety- five  of  column  two. 

The  ninety- five  in  column  two  are  the  top  ninety- five  selected 
by  the  board.  The  number  of  ratees  indicated  in  column  two  are 
grouped  according  to  the  number  of  responses  they  received ,  as 
indicated  in  column  one. 

The  number  of  ratees  indicated  in  column  three  are  grouped 
according  to  the  number  of  responses  they  received,  as  indicated 
in  column  one.  These  ratees  were  also  (along  with  the  respective 
ones  in  column  two)  in  the  top  ninety-five  as  determined  by 
the  peers. 


Column  1 

0 

1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 
20 


Column  2 

5 
11 
15 
12 

6 
6 
» 

6 

9 

5 

4 

4 

2 

1 

1 

1 


TOTAL  95 


Column  3 


6 

8 

4 

4 

4 

4 

•» 

f 

2 

2 

1 

2 

1 


45 


Percent  agreement  is  48 
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APPENDIX  G 


FREQUENCY  OF  RESPONSE  BY  PEERS  OF  TOP  NINETY-FIVE  SELECTED 
BY  PEERS  BASED  ON  259  NAMES  SUPPLIED  TO  THE  BOARD 


Column  one  indicates  the  number  of  responses  by  peers  on  the 
top  ninety- five  of  column  two. 

The  ninety- five  in  column  two  are  the  top  ninety- five  selected 
by  the  peers.  The  rumber  of  ratees  indicated  in  column  two  are 
grouped  according  to  the  number  of  responses  they  received,  as 
indicated  in  column  one. 

The  number  of  ratees  indicated  in  column  three  are  grouped 
according  to  the  number  of  responses  they  received,  as  indicated 
in  column  one.  These  ratees  were  also  (along  with  the  respective 
ones  in  column  two)  in  the  top  ninety- five  as  determined  by  the 
selection  board. 


Column  1 

1 

2 

3 

4 

5 

6 

7 

8 
9 

19 

11 

12 

13 

15 

17 


Ccluon  2 

18 

12 

13 

9 

7 

6 

7 

7 

4 

4 

2 

2 

? 

1 

1 


TOTAL  95 


Column  3 

6 

8 

4 

4 

4 


6 

2 

2 

1 

2 

1 


46 


Percent  agreement  is  48 
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APPENDIX  H 


FREQUENCY  OF  RESPONSE  BY  PEERS  OF  BOTTOM  NINETY-FIVE  SELECTED 
BY  PEERS  BASED  ON  259  NAICS  SUPPLIED  TO  THE  BOARD 


C  DluEn  one  indicates  the  number  of  responses  by  peers  on  the 
bottom  ninety- five  of  column  two. 

The  ninety- five  in  coIisja  two  are  the  bottom  ninety- five 
selected  by  the  peers.  The  number  of  ratees  i>  die a ted  in  column 
two  are  grouped  according  to  the  number  of  resjonses  they  received, 
as  indicated  in  column  one. 

The  number  of  ratees  indicated  in  colistn  three  are  grouped 
according  to  the  number  of  responses  they  receded.  indicated 
in  column  one.  These  ratees,  in  the  number  inoi  cited.  were  in  the 
top  ninety- five  as  determined  by  the  selection  < ard. 


Column  1 

0 

1 

2 

3 

4 

5 

6 

7 

8 
10 
20 


Column  2 

21 

21 

10 

13 

11 

? 

2 

3 

4 
2 
1 


tuuan  : 

5 

5 

3 

2 

<■> 

1 

1 

1 

1 

1 


TOTAL  95  20 


Percent  disagreement  is  21 
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APPENDIX  I 


FREQUENCY  OF  RESPONSE  BY  SUBORDINATES 
OF  OFFICERS  SELECTED  BY  THE  BOARD 


Column  one  indicates  the  number  of  responses  by  subordinates 
on  the  top  ninety-five  of  column  two. 

The  ninety- five  in  column  two  are  the  top  ninety- five  selected 
by  the  board.  The  number  of  ratees  indicated  in  column  two  are 
grouped  according  to  the  number  of  responses  they  received,  as 
indicated  in  column  one. 

The  number  of  ratees  indicated  in  column  three  are  grouped 
according  to  the  nwiber  of  responses  they  received,  as  indicated 
in  column  one.  Theje  ratees  were  also  (along  with  the  respective 
ones  in  column  two)  in  the  top  ninety- five  as  determined  by 
the  subordinates. 


Column  1 

0 

1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

15 

17 

18 
19 

23 

24 

25 


Column  2 

12 

19 

9 

12 

10 

6 

1 

1 

6 

4 

3 

2 

2 

1 

1 

1 

1 

1 

1 

1 

1 


TOTAL  95 


Column  3 


4 


4 
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APPENi*  a*  J 


FREQL'E'^Y  OF  RESPONSE  BY  SUBORDINATES  OF  TOP 
NINETY-FIVE  SELECTED  BY  SUBORDINATES 


Column  one  indicates  the  number  of  responses  by  subordinates 
on  the  top  ninety- five  of  column  two. 

The  ninety- five  in  column  two  are  the  top  ninety- five  selected 
by  the  subordinates.  The  number  of  ratees  indicated  in  column  two 
are  grouped  according  to  the  number  of  responses  r.hey  received,  as 
indicated  in  column  one. 

The  number  of  ratees  indicated  in  column  three  are  grouped 
according  to  the  number  of  responses;  they  received,  as  indicated 
in  column  one.  Ihese  ratees  were  also  (along  with  the  respective 
ones  in  column  two)  in  the  top  ninety- five  as  determined  by  the 
selection  board. 


Column  1 

Column  2 

Column  3 

1 

74 

4 

2 

17 

mm 

3 

4 

<■* 

TOTAL  95 

4 

APPENDIX  K 


FREQUENCY  OF  RESPONSE  BY  SUBORDINATES  OF  OFFICERS 
SELECTED  BY  THE  BOARD  BASED  ON  259  NAMES  SUPPLIED  TO  THF.  BOARD 


Column  one  indicates  the  number  of  responses  by  subordinates 
on  the  top  ninety- five  of  column  two. 

The  ninety- five  in  column  two  are  the  top  ninety- five  sel=rt2d 
by  the  board.  The  number  of  ratces  indicated  in  column  two  are 
grouped  according  to  the  number  of  responses  they  received,  as 
indicated  in  column  one. 

The  number  of  ratees  indicated  in  column  three  are  grouped 
according  to  the  number  of  responses  they  received,  as  indicated 
in  column  one.  These  ratees  were  also  (along  with  the  respective 
ones  in  column  two)  in  the  top  ninety- five  as  determined  by 
the  subordinates. 


Column  1 

0 

1 

2 

3 

4 

5 

6 
7 
3 
9 

10 

11 

12 

13 

15 

17 

18 
19 

23 

24 

25 


Column  ?. 

12 

19 

9 

12 

10 

6 

1 

1 

6 

4 

3 

2 

2 

1 

1 

1 

1 

1 

1 

1 

1 


TOTAL  95 


Column  3 


12 

5 

6 
6 
2 
1 

4 

1 

1 


1 

1 


40 


Percent  agreement  is  42 
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APPENDIX  L 


FREQUENCY  Cf  >’ONSE  BY  SUBORDINATES  OF  TOP  NINETY-FIVE 
SLUTTED  BY  SUBORDINATES  BASED  ON  259  NAMES  SUPPLIED  TO  THE  BOARD 


Column  one  indicates  the  number  of  responses  by  subordinates 
>  n  the  top  ninety- five  of  column  two. 

The  ninety-live  in  column  two  are  the  top  ninety-five  selected 
by  the  subordinates.  The  number  of  ratees  indicated  in  column  two 
are  grouped  according  to  the  number  of  responses  they  received,  as 
indicated  in  column  o:e. 

The  number  of  ratees  indicated  in  column  three  are  grouped 
according  to  the  number  of  responses  they  received,  as  indicated 
in  column  one.  These  ratees  were  also  (along  with  the  respective 
ones  in  column  two)  in  the  top  ninety-five-  as  determined  by  the 
selection  board. 


Column  1 

1 

2 

3 

4 

5 

6 
7 

ft 

q 

10 

11 

12 

14 

15 
19 


Column  2 

26 

17 

16 

8 

7 

3 

3 

5 

? 

3 

1 

l 


1 


Column  3 

12 

5 

8 

7 

2 

1 

4 

1 

2 

1 

1 

1 


TOTAL  95  45 


Percent  agreement  is  47 
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APPENDIX  M 


FREQUENCY  OF  RESPONSE  BY  SUBORDINATES  OF  BOTTOM  NINETY -FIVE 
SELECTED  BY  SUBORDINATES  BASED  ON  259  NAMES  SUPPLIED  TO  THE  BOARD 


Column  one  indicates  the  number  of  responses  by  subordinates 
on  the  bottom  ninety- five  of  column  two. 

T'ie  ninety- fiv.-  ir,  column  two  are  the  bottom  ninety-five 
selected  by  the  su’*ordinates.  The  number  of  ratees  indicated  in 
column  t;5©  are  grouped  according  to  the  number  of  responses  they 
received,  as  indicated  in  coleran  one. 

The  number  of  razees  indicated  in  column  three  are  grouped 
according  to  the  number  of  responses  they  received,  as  indicated 
in  column  one.  These  ratees,  in  the  number  indicated,  were  in  the 
top  ninety- five  ns  determined  Dy  the  selection  Hoard. 


Column  1 

0 

l 

4. 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

n 

18 

19 


Column  2 

37 

15 

6 

7 

6 

5 

5 

3 

3 

z 

1 

1 

1 

1 

1 

1 


Column  3 

12 

5 

1 


3 


1 

1 

1 

t 

1 

1 


TOTAL  95  29 


Percent  disagreement  is  30 
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APPENDIX  N 


COMPARISON  OF  PEER  RATED  TOP  NINETY-FIVE 
WITH  SUBORDINATE  RATINGS  FOR  THE  SAME  GROUP 


Column  one  indicates  cue  number  of  officers  the  peers  rated 
in  the  top  ninety- five;  column  two  indicates  the  numbex  of  responses 
which  they  received;  column  three  indicates  the  nudber  ot  responses 
for  the  same  group  from  the  subordinate  ratings;  column  four  indi¬ 
cates  the  number  of  officers  of  this  group  the  subordinates  also 
placed  ir.  the  top  ninety- five. 


Column  1 

64 

19 

5 

1 

2 

l 

1 

I 

• 

1 


Coluan  2 
0 
1 
2 

3 

4 

5 

6 

7 

8 
9 

12 

13 


Column  3 
25 


22 

9 

1.2 

8 

7 

5 

3 


Column  4 


11 

1 


1 


1 


15  1 

2:<  i 


95  TOTAL 


95 


12 


58 


APPENDIX  0 


COMPARISON  OF  PEER  AND  SUBORDINATE 
RANK  ORDER  OF  MERIT  ON  A  SELECTED  BASIS 


Column  one  is  the  peer  rank  order  of  merit  based  on  the 
number  of  ratings  in  parenthesis;  colon  two  is  the  subordinate 
rank  order  of  merit  based  on  the  number  of  ratings  in  parenthesis; 
column  three  indicates  if  the  ratee  was  in  the  2S9  names  submitted 
to  the  board. 


Column  1 

Column  2 

Column  3 

919  (15) 

495  (9) 

No 

792  (13) 

217  (8) 

No 

548  (10) 

822  (24) 

Yes* 

983  (10) 

6%  (8) 

No 

300  (8) 

714  (11) 

No 

16  (22) 

19  (47) 

No 

404  (11) 

546  (14) 

No 

667  (20) 

158  (8) 

Yes* 

313  (91 

455  (35) 

No 

831  (12) 

166  (12) 

No 

471  (14) 

451  (19) 

Yes* 

647  (10) 

697  (8) 

No 

233  (10) 

798  (16) 

Yes 

357  (11) 

592  (11) 

Yes* 

202  (12) 

710  (14) 

Yes 

451  (8) 

373  (12) 

No 

145  (11) 

354  (9) 

Yes* 

395  (10) 

604  (13) 

No 

160  (11) 

411  ill) 

Yes 

287  (9) 

719  (18) 

No 

316  (24) 

223  (10) 

Yes 

602  (9) 

231  (1.1) 

Yes* 

260  (12) 

603  (26) 

No 

199  (17) 

168  (19) 

Yes 

3S9  (8) 

844  (23) 

No 

263  (10) 

466  (23) 

No 

388  (11) 

888  (28) 

No 

520  (9) 

390  (17) 

No 

106  (8) 

594  (25) 

Yes* 

116  (8) 

181  (12) 

Yes* 

*Selected  W  the  board  in  the  top  ninety-five. 
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